Nov 04 03:46:28.984060 ip-10-0-129-253 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 04 03:46:28.984072 ip-10-0-129-253 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 04 03:46:28.984079 ip-10-0-129-253 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 04 03:46:28.984295 ip-10-0-129-253 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 04 03:46:39.201385 ip-10-0-129-253 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 04 03:46:39.201400 ip-10-0-129-253 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f21578e18a91401cb9f512df5163f8d3 --
Nov 04 03:48:49.743913 ip-10-0-129-253 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 03:48:50.075476 ip-10-0-129-253 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:48:50.075476 ip-10-0-129-253 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 03:48:50.075476 ip-10-0-129-253 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:48:50.075476 ip-10-0-129-253 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:48:50.079095 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079013    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:48:50.079095 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079094    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:48:50.079095 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079098    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079102    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079106    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079108    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079111    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079114    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079117    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079120    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079122    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079125    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079128    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079131    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079134    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079136    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079139    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079142    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079145    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079147    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079154    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079157    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:48:50.079187 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079159    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079162    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079167    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079171    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079174    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079177    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079180    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079182    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079185    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079188    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079190    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079193    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079196    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079199    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079201    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079204    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079207    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079212    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:48:50.079674 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079215    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079219    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079222    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079224    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079227    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079230    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079233    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079235    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079238    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079240    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079243    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079245    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079248    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079251    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079253    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079256    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079259    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079262    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079265    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079268    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:48:50.080099 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079270    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079273    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079275    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079278    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079280    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079283    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079286    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079289    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079291    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079294    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079296    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079299    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079302    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079304    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079307    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079310    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079312    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079315    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079317    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079320    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:48:50.080564 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079323    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079326    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079328    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079331    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079333    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079336    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079338    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079342    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079344    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079347    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079350    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079352    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079356    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079358    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079361    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079363    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079366    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079369    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079371    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079374    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:48:50.081058 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079376    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079379    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079381    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079384    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079386    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.079389    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080167    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080175    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080178    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080180    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080183    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080186    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080188    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080191    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080194    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080196    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080199    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080201    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080204    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080207    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:48:50.081534 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080210    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080212    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080215    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080217    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080220    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080223    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080225    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080228    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080231    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080233    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080236    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080238    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080241    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080244    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080246    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080249    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080251    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080254    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080257    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:48:50.082035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080261    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080265    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080268    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080271    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080273    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080276    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080279    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080281    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080284    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080286    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080289    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080291    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080294    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080297    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080300    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080302    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080305    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080308    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080310    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080313    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:48:50.082503 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080317    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080320    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080323    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080325    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080328    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080330    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080333    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080335    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080338    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080340    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080343    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080345    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080348    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080351    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080353    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080356    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080359    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080361    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080364    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:48:50.082990 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080366    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080369    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080371    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080374    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080376    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080380    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080384    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080386    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080389    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080392    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080394    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080398    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080403    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080406    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080409    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080413    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080416    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080418    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080421    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080423    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:48:50.083441 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080426    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080428    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080431    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080433    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080436    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080438    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080441    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080444    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080446    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080448    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080451    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080453    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080456    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080458    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080959    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080965    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080968    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080971    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080974    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:48:50.083937 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080977    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080980    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080983    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080985    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080988    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080992    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080995    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.080998    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081000    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081003    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081006    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081008    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081011    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081013    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081016    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081018    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081021    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081023    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081026    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:48:50.084397 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081029    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081031    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081034    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081036    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081039    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081041    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081044    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081047    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081050    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081052    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081055    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081058    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081060    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081063    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081066    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081068    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081071    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081074    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081076    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081079    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:48:50.084877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081082    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081084    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081087    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081091    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081094    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081096    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081099    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081101    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081104    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081106    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081109    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081112    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081115    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081118    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081121    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081124    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081126    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081129    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081132    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:48:50.085396 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081134    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081137    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081139    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081142    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081144    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081147    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081149    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081151    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081154    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081157    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081159    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081162    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081164    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081167    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081170    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081172    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081175    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081177    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081180    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081182    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:48:50.085951 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081185    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081188    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081190    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081193    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081195    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081197    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081200    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081206    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081209    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081211    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081214    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081216    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081219    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081221    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081224    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081227    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081229    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081232    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081234    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081236    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:48:50.086537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081239    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081244    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.081246    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081942    2573 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081951    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081958    2573 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081962    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081967    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081971    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081975    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081979    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081982    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081985    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081989    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081992    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081995    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.081998    2573 flags.go:64] FLAG: --cgroup-root=""
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082001    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082006    2573 flags.go:64] FLAG: --client-ca-file=""
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082009    2573 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082013    2573 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082018    2573 flags.go:64] FLAG: --cluster-domain=""
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082021    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 03:48:50.087056 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082024    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082027    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082031    2573 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082034    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082037    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082041    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082044    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082047    2573 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082051    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082054    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082057    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082060    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082065    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082068    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082071    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082074    2573 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082077    2573 flags.go:64] FLAG: --enable-server="true"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082080    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082084    2573 flags.go:64] FLAG: --event-burst="100"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082087    2573 flags.go:64] FLAG: --event-qps="50"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082090    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082093    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082096    2573 flags.go:64] FLAG: --eviction-hard=""
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082101    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 03:48:50.087597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082104    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082107    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082110    2573 flags.go:64] FLAG: --eviction-soft=""
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082114    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082117    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082121    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082124    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082127    2573 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082130    2573 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082133    2573 flags.go:64] FLAG: --feature-gates=""
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082137    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082140    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082143    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082146    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082149    2573 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082152    2573 flags.go:64] FLAG: --help="false"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082155    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082159    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082163    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082166    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082170    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082173    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082176    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 03:48:50.088208 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082179    2573 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082182    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082185    2573 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082188    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082191    2573 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082194    2573 flags.go:64] FLAG: --kube-reserved=""
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082197    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082200    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082203    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082206    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082209    2573 flags.go:64] FLAG: --lock-file=""
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082211    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082216    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082219    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082225    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082227    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082231    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082234    2573 flags.go:64] FLAG: --logging-format="text"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082237    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082241    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082244    2573 flags.go:64] FLAG: --manifest-url=""
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082247    2573 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082251    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082254    2573 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082258    2573 flags.go:64] FLAG: --max-pods="110"
Nov 04 03:48:50.088793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082261    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082264    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082267    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082270    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082274    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082277    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082280    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082288    2573 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082291    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082294    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082297    2573 flags.go:64] FLAG: --pod-cidr=""
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082300    2573 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082303    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082306    2573 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082309    2573 flags.go:64] FLAG: --port="10250"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082312    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082315    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-02ee2cea9e1beaecf"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082318    2573 flags.go:64] FLAG: --qos-reserved=""
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082321    2573 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082325    2573 flags.go:64] FLAG: --register-node="true"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082328    2573 flags.go:64] FLAG: --register-with-taints=""
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082332    2573 flags.go:64] FLAG: --registry-burst="10"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082335    2573 flags.go:64] FLAG: --registry-qps="5"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082339    2573 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082342    2573 flags.go:64] FLAG: --reserved-memory=""
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082346    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 03:48:50.089499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082349    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082352    2573 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082355    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082358    2573 flags.go:64] FLAG: --runonce="false"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082361    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082364    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082367    2573 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082370    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082373    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082376    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082379    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082382    2573 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082385    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082388    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082391    2573 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082394    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082397    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082400    2573 flags.go:64] FLAG: --system-cgroups=""
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082403    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082409    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082412    2573 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082415    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082419    2573 flags.go:64] FLAG: --tls-min-version=""
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082422    2573 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082425    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 03:48:50.090117 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082429    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082432    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082435    2573 flags.go:64] FLAG: --v="2"
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082440    2573 flags.go:64] FLAG: --version="false"
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082444    2573 flags.go:64] FLAG: --vmodule=""
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082448    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082452    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082591    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082595    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082599    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082602    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082605    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082608    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082610    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082613    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082628    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082631    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082634    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082637    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082639    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082642    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:48:50.090778 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082645    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082648    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082650    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082654    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082656    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082659    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082661    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082664    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082666    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082669    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082671    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082675    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082678    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082680    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082683    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082685    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082688    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082690    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082694    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:48:50.091268 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082696    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082699    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082702    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082704    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082707    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082709    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082712    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082716    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082718    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082721    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082723    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082727    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082732    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082735    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082738    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082740    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082743    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082745    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082748    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:48:50.091774 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082750    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082753    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082755    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082758    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082760    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082764    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082767    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082769    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082772    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082775    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082777    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082780    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082783    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082785    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082788    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082791    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082793    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082796    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082798    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:48:50.092229 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082802    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082808    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082810    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082813    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082816    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082818    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082821    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082824    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082826    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082829    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082831    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082834    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082837    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082839    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082842    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082844    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082847    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082850    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082853    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:48:50.092699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082856    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082859    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082861    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082864    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082866    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082869    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082871    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082874    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082877    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082879    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082881    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082884    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082886    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082889    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082893    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082895    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.082922    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:48:50.093144 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082957    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082961    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082964    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082967    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082970    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082973    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082976    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082979    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082982    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082984    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082987    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082989    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082992    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082995    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.082998    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083001    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083004    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083007    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083010    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:48:50.093544 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083013    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083015    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083018    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083021    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083023    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083027    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083030    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083033    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083035    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083038    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083040    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083045    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083048    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083051    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083053    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083056    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083058    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083061    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083063    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083066    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:48:50.093997 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083068    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083071    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083073    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083076    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083079    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083081    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083084    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083087    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083090    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083093    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083096    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083098    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083101    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083103    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083106    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083108    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083111    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083113    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083116    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083119    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:48:50.094476 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083121    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083124    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083126    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083130    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083133    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083136    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083138    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083141    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083143    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083146    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083148    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083151    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083153    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083156    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083158    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083161    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083166    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083171    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083174    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:48:50.095035 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083179    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083182    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083185    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083187    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083190    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083192    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083195    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083197    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083200    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083202    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083205    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083207    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083210    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083213    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083216    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083219    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083222    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083225    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083227    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:48:50.095477 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083230    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083233    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083235    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083238    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083240    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083244    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083248    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083251    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.083254    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.090466    2573 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.090480    2573 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.090497    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.090502    2573 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090579    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090585    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090589    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090592    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090595    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:48:50.095943 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090598    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090601    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090604    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090607    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090609    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090612    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090614    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090644    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090648    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090650    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090653    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090656    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090659    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090661    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090664    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090667    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090669    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090672    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090675    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090678    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:48:50.096386 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090681    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090683    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090686    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090688    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090692    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090695    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090699    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090701    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090703    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090706    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090709    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090711    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090714    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090716    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090719    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090721    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090724    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090727    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090729    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090732    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:48:50.096899 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090740    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090743    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090746    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090748    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090750    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090753    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090755    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090758    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090761    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090763    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090766    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090769    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090771    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090774    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090776    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090779    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090782    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090785    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090788    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090791    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:48:50.097537 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090793    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090796    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090799    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090801    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090804    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090806    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090809    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090812    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090814    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090819    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090823    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090826    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090829    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090837    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090840    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090842    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090845    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090847    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090851    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:48:50.098877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090855    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090859    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090862    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090866    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090868    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090871    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090874    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090876    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090879    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090882    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090885    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090887    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090890    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090892    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090895    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090898    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090906    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090909    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090911    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:48:50.099331 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090914    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090916    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.090919    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.090946    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091141    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091146    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091150    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091155    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091164    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091167    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091170    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091173    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091175    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091178    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091180    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091183    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:48:50.099810 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091186    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091188    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091190    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091193    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091195    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091199    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091201    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091204    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091206    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091209    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091211    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091214    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091217    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091219    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091222    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091224    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091227    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091229    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091232    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091234    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:48:50.100201 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091237    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091240    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091242    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091244    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091247    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091255    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091258    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091261    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091263    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091266    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091269    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091272    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091274    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091277    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091280    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091282    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091285    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091288    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091290    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:48:50.100699 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091293    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091295    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091298    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091300    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091303    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091305    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091308    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091310    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091313    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091315    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091318    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091320    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091323    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091325    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091328    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091330    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091333    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091335    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091338    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:48:50.101290 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091346    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091350    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091354    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091358    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091361    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091364    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091367    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091370    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091372    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091375    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091377    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091380    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091383    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091385    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091388    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091391    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091393    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091395    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091398    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:48:50.101852 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091400    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091403    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091406    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091408    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091411    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091413    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091416    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091419    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091421    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091424    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091426    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091429    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091431    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091434    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091442    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091445    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:48:50.102309 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091448    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.091472    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091514    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091518    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091520    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091523    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091526    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091528    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091531    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091534    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091536    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091539    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091541    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091544    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091546    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091549    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:48:50.102950 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091551    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091554    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091556    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091559    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091561    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091564    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091566    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091568    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091571    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091573    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091576    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091578    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091581    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091583    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091585    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091595    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091598    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091601    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091603    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:48:50.103391 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091606    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091609    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091611    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091614    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091629    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091633    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091636    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091639    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091641    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091644    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091646    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091650    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091654    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091657    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091660    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091663    2573 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091666    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091669    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091671    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091674    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:48:50.103851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091676    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091679    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091682    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091684    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091687    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091690    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091692    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091695    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091697    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091706    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091709    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091712    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091715    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091717    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091720    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091723    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091725    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091728    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091731    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:48:50.104317 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091733    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091736    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091738    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091741    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091744    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091746    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091749    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091751    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091754    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091757    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091761    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091764    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091767    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091769    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091772    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091775    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091777    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091780    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091783    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:48:50.104809 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091785    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091788    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091791    2573 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091793    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091801    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091804    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091806    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091809    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091811    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091814    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091817    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091819    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091827    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091829    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:50.091832    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.091889    2573 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.093774    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.094580    2573 server.go:1014] "Starting client certificate rotation"
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.094679    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:48:50.105295 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.095367    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:48:50.105779 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.096300    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:48:50.105779 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.101946    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:48:50.111267 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.111250    2573 log.go:25] "Validated CRI v1 runtime API"
Nov 04 03:48:50.114079 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.114063    2573 log.go:25] "Validated CRI v1 image API"
Nov 04 03:48:50.114910 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.114871    2573 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 03:48:50.120156 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.120139    2573 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aaa51dfe-3927-45e7-b95d-c052a1b66fda:/dev/nvme0n1p3 e8155bf8-bfe0-4ce0-97ca-af858267fde6:/dev/nvme0n1p4]
Nov 04 03:48:50.120233 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.120156    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 03:48:50.125324 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.125218    2573 manager.go:217] Machine: {Timestamp:2026-11-04 03:48:50.123991029 +0000 UTC m=+0.283045882 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100323 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a2a041c1785097e0809f4a0a68f59 SystemUUID:ec2a2a04-1c17-8509-7e08-09f4a0a68f59 BootID:f21578e1-8a91-401c-b9f5-12df5163f8d3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fa:31:dc:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fa:31:dc:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:24:53:92:a1:bf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 03:48:50.125324 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.125323    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 03:48:50.125435 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.125409    2573 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 03:48:50.126835 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.126810    2573 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 03:48:50.126970 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.126836    2573 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-253.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 03:48:50.127018 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.126973    2573 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 03:48:50.127018 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.126981    2573 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 03:48:50.127018 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.127006    2573 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:48:50.127094 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.127020    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:48:50.127701 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.127690    2573 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 03:48:50.131046 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.131034    2573 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 03:48:50.131181 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.131172    2573 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 03:48:50.131217 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.131201    2573 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 03:48:50.131217 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.131212    2573 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 03:48:50.131276 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.131224    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 03:48:50.131276 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.131234    2573 kubelet.go:405] "Adding apiserver pod source"
Nov 04 03:48:50.131276 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.131244    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 03:48:50.132233 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.132209    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:48:50.132357 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.132332    2573 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 03:48:50.136836 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.136251    2573 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 03:48:50.141312 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.141295    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:48:50.142039 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142025    2573 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 03:48:50.142103 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142066    2573 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 03:48:50.142247 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142234    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 03:48:50.142304 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142254    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 03:48:50.142304 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142264    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 03:48:50.142304 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142274    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 03:48:50.142304 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142283    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 03:48:50.142304 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142294    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 03:48:50.142834 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142823    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 03:48:50.142883 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142837    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 03:48:50.142883 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142850    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 03:48:50.142883 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142860    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 03:48:50.143012 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142899    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 03:48:50.143012 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.142917    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 03:48:50.143846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.143835    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 03:48:50.143896 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.143853    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 03:48:50.146226 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.146190    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-253.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 03:48:50.147714 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.147698    2573 server.go:1263] "Started kubelet"
Nov 04 03:48:50.147787 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.147754    2573 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 03:48:50.147977 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.147922    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 03:48:50.148041 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.148008    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 03:48:50.148584 ip-10-0-129-253 systemd[1]: Started Kubernetes Kubelet.
Nov 04 03:48:50.148994 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.148951    2573 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 03:48:50.149843 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.149829    2573 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 03:48:50.151922 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.151903    2573 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-253.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 04 03:48:50.152053 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.152036    2573 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 04 03:48:50.153579 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.153559    2573 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 03:48:50.153775 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.152039    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-253.us-east-2.compute.internal.18e4baf8fc4a2205  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-253.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-253.us-east-2.compute.internal,},FirstTimestamp:2026-11-04 03:48:50.147672581 +0000 UTC m=+0.306727433,LastTimestamp:2026-11-04 03:48:50.147672581 +0000 UTC m=+0.306727433,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-253.us-east-2.compute.internal,}"
Nov 04 03:48:50.154760 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.154742    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hnscj"
Nov 04 03:48:50.154826 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.154760    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 03:48:50.154986 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.154936    2573 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 03:48:50.155130 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.155115    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:50.155999 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.155978    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 03:48:50.156082 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156011    2573 factory.go:55] Registering systemd factory
Nov 04 03:48:50.156082 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156030    2573 factory.go:223] Registration of the systemd container factory successfully
Nov 04 03:48:50.156181 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156096    2573 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 03:48:50.156181 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156112    2573 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 03:48:50.156277 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156234    2573 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 03:48:50.156277 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156244    2573 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 03:48:50.156577 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156557    2573 factory.go:153] Registering CRI-O factory
Nov 04 03:48:50.156577 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156577    2573 factory.go:223] Registration of the crio container factory successfully
Nov 04 03:48:50.156711 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156645    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 03:48:50.156711 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156663    2573 factory.go:103] Registering Raw factory
Nov 04 03:48:50.156711 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.156676    2573 manager.go:1199] Started watching for new ooms in manager
Nov 04 03:48:50.157527 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.157511    2573 manager.go:319] Starting recovery of all containers
Nov 04 03:48:50.162305 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.162270    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 03:48:50.164776 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.164743    2573 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-253.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 03:48:50.168508 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.168482    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hnscj"
Nov 04 03:48:50.168900 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.168872    2573 manager.go:324] Recovery completed
Nov 04 03:48:50.171542 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.171503    2573 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 04 03:48:50.176068 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.176054    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:48:50.177927 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.177907    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:48:50.178005 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.177942    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:48:50.178005 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.177957    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:48:50.178531 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.178517    2573 cpu_manager.go:226] "Starting" policy="none"
Nov 04 03:48:50.178531 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.178527    2573 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 03:48:50.178531 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.178528    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:48:50.178750 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.178541    2573 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 03:48:50.179233 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.179216    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:48:50.179307 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.179242    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:48:50.179307 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.179252    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:48:50.180562 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.180481    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-253.us-east-2.compute.internal.18e4baf8fe17bb99  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-253.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-253.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-253.us-east-2.compute.internal,},FirstTimestamp:2026-11-04 03:48:50.177923993 +0000 UTC m=+0.336978846,LastTimestamp:2026-11-04 03:48:50.177923993 +0000 UTC m=+0.336978846,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-253.us-east-2.compute.internal,}"
Nov 04 03:48:50.256244 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.256206    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:50.320078 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.320057    2573 policy_none.go:50] "Start"
Nov 04 03:48:50.320482 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.320472    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 03:48:50.320527 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.320493    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 03:48:50.322596 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.322577    2573 policy_none.go:44] "Start"
Nov 04 03:48:50.350312 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.350132    2573 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 03:48:50.354729 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.354714    2573 manager.go:341] "Starting Device Plugin manager"
Nov 04 03:48:50.355119 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.355104    2573 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 03:48:50.355119 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.355118    2573 server.go:87] "Starting device plugin registration server"
Nov 04 03:48:50.355347 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.355335    2573 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 03:48:50.355411 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.355348    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 03:48:50.355468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.355450    2573 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 03:48:50.355550 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.355532    2573 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 03:48:50.355550 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.355545    2573 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 03:48:50.356143 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.356122    2573 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 03:48:50.356215 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.356172    2573 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:50.385315 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.371008    2573 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-253.us-east-2.compute.internal\" not found" node="ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.456217 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.456165    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:48:50.457255 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.457241    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:48:50.457319 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.457274    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:48:50.457319 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.457289    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:48:50.457319 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.457319    2573 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.465631 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.465604    2573 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.465715 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.465640    2573 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-253.us-east-2.compute.internal\": node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:50.476582 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.476561    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 03:48:50.476582 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.476586    2573 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 03:48:50.476734 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.476606    2573 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 03:48:50.476734 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.476660    2573 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 03:48:50.484973 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.484953    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:50.491311 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.491298    2573 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:48:50.577192 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.577144    2573 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"]
Nov 04 03:48:50.577367 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.577226    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:48:50.578326 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.578311    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:48:50.578390 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.578342    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:48:50.578390 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.578358    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:48:50.580669 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.580656    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:48:50.580828 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.580814    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.580871 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.580844    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:48:50.581771 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.581724    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:48:50.581771 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.581745    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:48:50.581771 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.581754    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:48:50.581894 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.581780    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:48:50.581894 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.581791    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:48:50.581894 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.581760    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:48:50.583846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.583831    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.583893 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.583859    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:48:50.585029 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.585013    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:50.585207 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.585189    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:48:50.585308 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.585215    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:48:50.585308 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.585227    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:48:50.630383 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.630363    2573 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found" node="ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.633227 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.633212    2573 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found" node="ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.659679 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.659650    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f909644ddda3f20f4057bfd2523b2509-config\") pod \"kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"f909644ddda3f20f4057bfd2523b2509\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.659774 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.659682    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.659774 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.659706    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.685584 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.685547    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:50.760143 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.760103    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f909644ddda3f20f4057bfd2523b2509-config\") pod \"kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"f909644ddda3f20f4057bfd2523b2509\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.760143 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.760140    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.760339 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.760161    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.760339 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.760205    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f909644ddda3f20f4057bfd2523b2509-config\") pod \"kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"f909644ddda3f20f4057bfd2523b2509\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.760339 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.760232    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.760339 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.760208    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.785925 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.785897    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:50.886802 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.886727    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:50.932928 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.932901    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.935922 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:50.935905    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:50.987024 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:50.987000    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:51.087467 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:51.087446    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:51.094602 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.094582    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 03:48:51.094741 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.094717    2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 03:48:51.173296 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.173246    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 03:43:50 +0000 UTC" deadline="2028-07-05 16:15:36.276060981 +0000 UTC"
Nov 04 03:48:51.173296 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.173268    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14628h26m45.10279612s"
Nov 04 03:48:51.188431 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:51.188416    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:51.288934 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:51.288907    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:51.322546 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.322523    2573 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:48:51.390004 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:51.389977    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:51.490849 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:51.490801    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:51.525848 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.525819    2573 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:48:51.591940 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:51.591910    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:51.633240 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:51.633198    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf909644ddda3f20f4057bfd2523b2509.slice/crio-a4d63b7b1d745fea45bfcef6030a7b315629c44a9f1861a86b5fa8a5e9870456 WatchSource:0}: Error finding container a4d63b7b1d745fea45bfcef6030a7b315629c44a9f1861a86b5fa8a5e9870456: Status 404 returned error can't find the container with id a4d63b7b1d745fea45bfcef6030a7b315629c44a9f1861a86b5fa8a5e9870456
Nov 04 03:48:51.639371 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.639154    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:48:51.644283 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:51.644262    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfecc599dcc2092a8cb017231452a28ae.slice/crio-9df2c3649bb0878ed89a04f42679d1ef8f81eb62a749d84e255daccabfeede60 WatchSource:0}: Error finding container 9df2c3649bb0878ed89a04f42679d1ef8f81eb62a749d84e255daccabfeede60: Status 404 returned error can't find the container with id 9df2c3649bb0878ed89a04f42679d1ef8f81eb62a749d84e255daccabfeede60
Nov 04 03:48:51.692852 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:51.692818    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-253.us-east-2.compute.internal\" not found"
Nov 04 03:48:51.737792 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.737767    2573 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:48:51.756209 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.756158    2573 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:51.768912 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.768899    2573 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:48:51.770053 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.770040    2573 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:48:51.781581 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.781567    2573 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:48:51.963899 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:51.963882    2573 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:48:52.132963 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.132904    2573 apiserver.go:52] "Watching apiserver"
Nov 04 03:48:52.144016 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.143678    2573 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 03:48:52.144869 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.144842    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-lw2sq","openshift-ovn-kubernetes/ovnkube-node-89gb5","kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-95jzm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal","openshift-multus/network-metrics-daemon-hn8bx","kube-system/global-pull-secret-syncer-ksb9x","openshift-dns/node-resolver-5n7tp","openshift-network-operator/iptables-alerter-wmvzs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r","openshift-image-registry/node-ca-7jmdr","openshift-multus/multus-additional-cni-plugins-vqkzw","openshift-network-diagnostics/network-check-target-g97j2","kube-system/konnectivity-agent-49hmt"]
Nov 04 03:48:52.149049 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.149031    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.151301 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.151282    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:52.151391 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.151343    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:48:52.153268 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.153238    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 03:48:52.153268 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.153244    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 03:48:52.153928 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.153387    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x8rgd\""
Nov 04 03:48:52.153928 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.153543    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.155381 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.155364    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 03:48:52.156210 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.156025    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.158888 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.158870    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.161919 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.161899    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:52.162008 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.161973    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:48:52.164434 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.164413    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:52.164506 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.164488    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:48:52.166743 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.166719    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-hosts-file\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.166852 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.166748    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgx6l\" (UniqueName: \"kubernetes.io/projected/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-kube-api-access-cgx6l\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.166852 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.166772    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.166852 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.166809    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysconfig\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167007 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.166875    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-registration-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.167007 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.166919    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.167007 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.166949    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-iptables-alerter-script\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.167007 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.166991    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-run\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167185 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167037    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-tmp\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167185 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167128    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-host-slash\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.167185 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167155    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-kubernetes\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167185 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167169    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-var-lib-kubelet\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167185 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167184    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-modprobe-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167403 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167259    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-device-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.167403 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167309    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-sys-fs\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.167403 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167354    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-tmp-dir\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.167403 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167392    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-socket-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.167615 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167414    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167615 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167515    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-systemd\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167615 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167553    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.167615 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167580    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-conf\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167615 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167607    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-sys\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167885 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167634    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-tuned\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.167885 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167780    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:52.167885 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167817    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sjlf\" (UniqueName: \"kubernetes.io/projected/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-kube-api-access-5sjlf\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.167885 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167843    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-host\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.168095 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167944    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vljp6\" (UniqueName: \"kubernetes.io/projected/0982ff5a-da06-4180-9838-6bfc5f2deba0-kube-api-access-vljp6\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.168095 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.167993    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-lib-modules\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.168095 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.168058    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qpr7\" (UniqueName: \"kubernetes.io/projected/86f60625-8953-4f46-aa16-cc1d9142c4df-kube-api-access-2qpr7\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.169694 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.169676    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.172236 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.172216    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.174636 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.174616    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.175648 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.175626    2573 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 03:48:52.178015 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.177993    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:48:52.196583 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.196565    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4wlvp"
Nov 04 03:48:52.209518 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.209492    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4wlvp"
Nov 04 03:48:52.257138 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.257113    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 03:48:52.261791 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.261745    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:48:52.261791 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.261790    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 03:48:52.262129 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.262104    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 03:48:52.262129 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.262112    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 03:48:52.262248 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.262116    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 03:48:52.262248 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.262166    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 03:48:52.262248 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.262190    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 03:48:52.262248 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.262195    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 03:48:52.262248 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.262215    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 03:48:52.262501 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.262346    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 03:48:52.263554 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.263537    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 03:48:52.263664 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.263585    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 03:48:52.263721 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.263699    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4ttlw\""
Nov 04 03:48:52.263794 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.263779    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:48:52.263859 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.263835    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 03:48:52.263913 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.263888    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 03:48:52.263968 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.263938    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 03:48:52.264285 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.264267    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-twz4j\""
Nov 04 03:48:52.265172 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.264998    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 03:48:52.265172 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265064    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-75zvg\""
Nov 04 03:48:52.265172 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265072    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 03:48:52.265172 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265068    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j5cz2\""
Nov 04 03:48:52.265172 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265119    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-78gsl\""
Nov 04 03:48:52.265172 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265152    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 03:48:52.265568 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265315    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tl6ng\""
Nov 04 03:48:52.265568 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265354    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m5nwx\""
Nov 04 03:48:52.265568 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265381    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 03:48:52.265568 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265412    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 03:48:52.265568 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265442    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 03:48:52.265568 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265460    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 03:48:52.265568 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265476    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-qdtd9\""
Nov 04 03:48:52.265568 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265386    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 03:48:52.265568 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.265518    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 03:48:52.268251 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268229    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-lib-modules\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.268348 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268270    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-hosts-file\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.268404 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268342    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-hosts-file\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.268404 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268383    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgx6l\" (UniqueName: \"kubernetes.io/projected/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-kube-api-access-cgx6l\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.268404 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268396    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-lib-modules\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.268560 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268421    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-multus\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.268560 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268451    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drvtd\" (UniqueName: \"kubernetes.io/projected/b9b6d0b0-68ba-4d89-9476-2727609a1264-kube-api-access-drvtd\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.268560 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268500    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.268560 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268528    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.268560 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268557    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.268846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268581    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-run\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.268846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268606    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-kubelet\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.268846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268665    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-var-lib-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.268846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268712    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-etc-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.268846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268739    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-host-slash\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.268846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268782    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.268846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268801    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-var-lib-kubelet\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.268846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268816    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-run\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.268846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268830    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-host-slash\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.269260 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268863    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zfjv\" (UniqueName: \"kubernetes.io/projected/d4639182-9c75-4c83-81aa-e8e59d52fe6e-kube-api-access-4zfjv\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.269260 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268890    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:52.269260 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.268936    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-var-lib-kubelet\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.269260 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269049    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-system-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.269260 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269075    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-bin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.269260 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269098    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-netd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.269260 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269121    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-dbus\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:52.269260 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269174    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-cnibin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.269260 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269225    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovn-node-metrics-cert\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.269260 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269258    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-script-lib\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269285    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-device-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269326    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-sys-fs\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269351    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-socket-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269374    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269400    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6bb8b1d-e87b-4167-a84a-92e9815f2206-konnectivity-ca\") pod \"konnectivity-agent-49hmt\" (UID: \"d6bb8b1d-e87b-4167-a84a-92e9815f2206\") " pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269436    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269460    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-netns\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269516    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-binary-copy\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269561    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-systemd\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269588    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-cni-binary-copy\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269613    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269640    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269684    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-log-socket\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269759    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-sys\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269784    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-tuned\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269814    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6bb8b1d-e87b-4167-a84a-92e9815f2206-agent-certs\") pod \"konnectivity-agent-49hmt\" (UID: \"d6bb8b1d-e87b-4167-a84a-92e9815f2206\") " pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:48:52.270341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269839    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-etc-kubernetes\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269864    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87kzd\" (UniqueName: \"kubernetes.io/projected/a021abf7-aecb-4044-8f56-86038fd77b5b-kube-api-access-87kzd\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269889    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-system-cni-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269910    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-config\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269943    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269959    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sjlf\" (UniqueName: \"kubernetes.io/projected/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-kube-api-access-5sjlf\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269974    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-host\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.269995    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vljp6\" (UniqueName: \"kubernetes.io/projected/0982ff5a-da06-4180-9838-6bfc5f2deba0-kube-api-access-vljp6\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270011    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-kubelet-config\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270026    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-socket-dir-parent\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270050    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270071    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qpr7\" (UniqueName: \"kubernetes.io/projected/86f60625-8953-4f46-aa16-cc1d9142c4df-kube-api-access-2qpr7\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270262    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysconfig\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270286    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-sys\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270314    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-multus-certs\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270349    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-slash\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270378    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-ovn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.270971 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270403    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-env-overrides\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270432    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-registration-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270453    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-iptables-alerter-script\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270472    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-tmp\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270495    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270517    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-kubernetes\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270534    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-modprobe-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270556    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4639182-9c75-4c83-81aa-e8e59d52fe6e-serviceca\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270573    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-k8s-cni-cncf-io\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270582    2573 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270621    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-kubelet\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270647    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-bin\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270677    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-tmp-dir\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270715    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsqvw\" (UniqueName: \"kubernetes.io/projected/37e230e6-a451-475a-8b79-9d000021644b-kube-api-access-bsqvw\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270742    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-conf-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270769    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-daemon-config\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270796    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-systemd-units\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.271641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270824    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-node-log\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270849    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4639182-9c75-4c83-81aa-e8e59d52fe6e-host\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270877    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270904    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-systemd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270932    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.270983    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-conf\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271011    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-cnibin\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271040    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdbcn\" (UniqueName: \"kubernetes.io/projected/0bb68137-3e53-42d0-a9fd-e6cddab3346c-kube-api-access-jdbcn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271067    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-hostroot\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271130    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-host\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271154    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-netns\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271267    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-kubernetes\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271359    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-modprobe-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271737    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-tmp-dir\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271833    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271863    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-device-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271918    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-sys-fs\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.272463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.271953    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-conf\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.273299 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.272017    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-socket-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.273299 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.272091    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-systemd\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.273299 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.272129    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysconfig\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.273299 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.272144    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.273299 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.272182    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-registration-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.273299 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.272672    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-iptables-alerter-script\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.274618 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.274595    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-tuned\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.274709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.274693    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-tmp\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.281427 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.281397    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgx6l\" (UniqueName: \"kubernetes.io/projected/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-kube-api-access-cgx6l\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.285002 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.284983    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:48:52.285105 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.285007    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:48:52.285105 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.285022    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ggjv6 for pod openshift-network-diagnostics/network-check-target-g97j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:48:52.285105 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.285080    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6 podName:4bbeb235-acfa-4f82-8edb-dc0727347f40 nodeName:}" failed. No retries permitted until 2026-11-04 03:48:52.785055929 +0000 UTC m=+2.944110789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ggjv6" (UniqueName: "kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6") pod "network-check-target-g97j2" (UID: "4bbeb235-acfa-4f82-8edb-dc0727347f40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:48:52.286137 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.286108    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vljp6\" (UniqueName: \"kubernetes.io/projected/0982ff5a-da06-4180-9838-6bfc5f2deba0-kube-api-access-vljp6\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.287313 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.287289    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sjlf\" (UniqueName: \"kubernetes.io/projected/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-kube-api-access-5sjlf\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.287795 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.287774    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qpr7\" (UniqueName: \"kubernetes.io/projected/86f60625-8953-4f46-aa16-cc1d9142c4df-kube-api-access-2qpr7\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.371529 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371500    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-systemd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.371692 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371542    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-cnibin\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.371692 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371569    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdbcn\" (UniqueName: \"kubernetes.io/projected/0bb68137-3e53-42d0-a9fd-e6cddab3346c-kube-api-access-jdbcn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.371692 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371594    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-hostroot\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.371692 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371621    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-netns\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.371920 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371738    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-multus\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.371920 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371768    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drvtd\" (UniqueName: \"kubernetes.io/projected/b9b6d0b0-68ba-4d89-9476-2727609a1264-kube-api-access-drvtd\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.371920 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371778    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-hostroot\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.371920 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371808    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.371920 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371859    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-multus\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.371920 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371864    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-cnibin\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.372209 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372101    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372283 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372260    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-systemd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372337 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.371835    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372337 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372311    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372337 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372301    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-netns\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372448 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372333    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-kubelet\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372448 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372375    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-var-lib-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372448 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372399    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-etc-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372448 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372403    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-kubelet\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372448 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372420    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zfjv\" (UniqueName: \"kubernetes.io/projected/d4639182-9c75-4c83-81aa-e8e59d52fe6e-kube-api-access-4zfjv\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.372448 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372439    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:52.372448 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372438    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-var-lib-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372451    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-etc-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372474    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-system-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372498    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-bin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372522    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-netd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.372548    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372552    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-system-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372592    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-netd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372596    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-bin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.372605    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs podName:37e230e6-a451-475a-8b79-9d000021644b nodeName:}" failed. No retries permitted until 2026-11-04 03:48:52.872591921 +0000 UTC m=+3.031646765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs") pod "network-metrics-daemon-hn8bx" (UID: "37e230e6-a451-475a-8b79-9d000021644b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372548    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-dbus\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372675    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-cnibin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372702    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovn-node-metrics-cert\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372728    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-script-lib\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372734    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-dbus\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372757    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6bb8b1d-e87b-4167-a84a-92e9815f2206-konnectivity-ca\") pod \"konnectivity-agent-49hmt\" (UID: \"d6bb8b1d-e87b-4167-a84a-92e9815f2206\") " pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:48:52.372809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372797    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-cnibin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372826    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372845    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-netns\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372862    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-binary-copy\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372879    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-cni-binary-copy\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372900    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372917    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372934    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-log-socket\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372951    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6bb8b1d-e87b-4167-a84a-92e9815f2206-agent-certs\") pod \"konnectivity-agent-49hmt\" (UID: \"d6bb8b1d-e87b-4167-a84a-92e9815f2206\") " pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372967    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-etc-kubernetes\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.372988    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87kzd\" (UniqueName: \"kubernetes.io/projected/a021abf7-aecb-4044-8f56-86038fd77b5b-kube-api-access-87kzd\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373005    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-system-cni-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373021    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-config\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373056    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-kubelet-config\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373084    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-socket-dir-parent\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373110    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373124    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.373709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373137    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-multus-certs\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373164    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-slash\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373169    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-netns\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373191    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-ovn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373217    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-env-overrides\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373247    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373273    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4639182-9c75-4c83-81aa-e8e59d52fe6e-serviceca\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373299    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-k8s-cni-cncf-io\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373326    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-kubelet\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373353    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-bin\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373383    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsqvw\" (UniqueName: \"kubernetes.io/projected/37e230e6-a451-475a-8b79-9d000021644b-kube-api-access-bsqvw\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373384    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6bb8b1d-e87b-4167-a84a-92e9815f2206-konnectivity-ca\") pod \"konnectivity-agent-49hmt\" (UID: \"d6bb8b1d-e87b-4167-a84a-92e9815f2206\") " pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373407    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-conf-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373434    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-daemon-config\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373464    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-systemd-units\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373460    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-multus-certs\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373507    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-node-log\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373535    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4639182-9c75-4c83-81aa-e8e59d52fe6e-host\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.374593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373545    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-log-socket\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373563    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373661    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-script-lib\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373776    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-binary-copy\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373947    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.373984    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-kubelet\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374049    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-slash\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374091    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374105    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-ovn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374155    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-bin\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.374197    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374209    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-etc-kubernetes\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.374253    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret podName:e38ea962-08ee-44b8-8872-feec06d36ada nodeName:}" failed. No retries permitted until 2026-11-04 03:48:52.874235866 +0000 UTC m=+3.033290724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret") pod "global-pull-secret-syncer-ksb9x" (UID: "e38ea962-08ee-44b8-8872-feec06d36ada") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374318    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-cni-binary-copy\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374376    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-system-cni-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374437    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-socket-dir-parent\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374469    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-config\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.375468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374549    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-kubelet-config\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:52.376194 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374580    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-env-overrides\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.376194 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374637    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-systemd-units\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.376194 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374678    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4639182-9c75-4c83-81aa-e8e59d52fe6e-serviceca\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.376194 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374683    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-node-log\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.376194 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374744    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.376194 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374750    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4639182-9c75-4c83-81aa-e8e59d52fe6e-host\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.376194 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.374782    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-k8s-cni-cncf-io\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.376194 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.375264    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.376194 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.375378    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-conf-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.376194 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.375566    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-daemon-config\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.377330 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.377308    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6bb8b1d-e87b-4167-a84a-92e9815f2206-agent-certs\") pod \"konnectivity-agent-49hmt\" (UID: \"d6bb8b1d-e87b-4167-a84a-92e9815f2206\") " pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:48:52.379494 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.379471    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovn-node-metrics-cert\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.389353 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.389292    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drvtd\" (UniqueName: \"kubernetes.io/projected/b9b6d0b0-68ba-4d89-9476-2727609a1264-kube-api-access-drvtd\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.390348 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.390327    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdbcn\" (UniqueName: \"kubernetes.io/projected/0bb68137-3e53-42d0-a9fd-e6cddab3346c-kube-api-access-jdbcn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.390430 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.390348    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsqvw\" (UniqueName: \"kubernetes.io/projected/37e230e6-a451-475a-8b79-9d000021644b-kube-api-access-bsqvw\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:52.390430 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.390352    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zfjv\" (UniqueName: \"kubernetes.io/projected/d4639182-9c75-4c83-81aa-e8e59d52fe6e-kube-api-access-4zfjv\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.391111 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.391089    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87kzd\" (UniqueName: \"kubernetes.io/projected/a021abf7-aecb-4044-8f56-86038fd77b5b-kube-api-access-87kzd\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.461832 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.461800    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:48:52.472576 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:52.472545    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc23bbb_cc49_4e50_a94a_df074fcd4d3e.slice/crio-3f25a6490432e4fec55af20c26b754a466bacdfec844667aec4125087850f471 WatchSource:0}: Error finding container 3f25a6490432e4fec55af20c26b754a466bacdfec844667aec4125087850f471: Status 404 returned error can't find the container with id 3f25a6490432e4fec55af20c26b754a466bacdfec844667aec4125087850f471
Nov 04 03:48:52.473360 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.473345    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:48:52.478545 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.478524    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:48:52.482622 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.482567    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5n7tp" event={"ID":"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e","Type":"ContainerStarted","Data":"3f25a6490432e4fec55af20c26b754a466bacdfec844667aec4125087850f471"}
Nov 04 03:48:52.483960 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:52.483937    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f60625_8953_4f46_aa16_cc1d9142c4df.slice/crio-36508a0c4387cbd148e95192f28fdfd39b34a9db94d58ac0e986eac0b68285cc WatchSource:0}: Error finding container 36508a0c4387cbd148e95192f28fdfd39b34a9db94d58ac0e986eac0b68285cc: Status 404 returned error can't find the container with id 36508a0c4387cbd148e95192f28fdfd39b34a9db94d58ac0e986eac0b68285cc
Nov 04 03:48:52.484061 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.483997    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:48:52.484725 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.484493    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal" event={"ID":"fecc599dcc2092a8cb017231452a28ae","Type":"ContainerStarted","Data":"9df2c3649bb0878ed89a04f42679d1ef8f81eb62a749d84e255daccabfeede60"}
Nov 04 03:48:52.486037 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.486011    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal" event={"ID":"f909644ddda3f20f4057bfd2523b2509","Type":"ContainerStarted","Data":"a4d63b7b1d745fea45bfcef6030a7b315629c44a9f1861a86b5fa8a5e9870456"}
Nov 04 03:48:52.491781 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.491758    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lw2sq"
Nov 04 03:48:52.495039 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:52.495012    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931e1236_fd6a_40f0_bd32_6e2f217eb5ff.slice/crio-594c966f892813491d1f7cc9e017421c2dc1870030368c603635501ced8aa703 WatchSource:0}: Error finding container 594c966f892813491d1f7cc9e017421c2dc1870030368c603635501ced8aa703: Status 404 returned error can't find the container with id 594c966f892813491d1f7cc9e017421c2dc1870030368c603635501ced8aa703
Nov 04 03:48:52.495793 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:52.495771    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0982ff5a_da06_4180_9838_6bfc5f2deba0.slice/crio-df2b256b863f2a77521ffb41b85876cf1da603e33af7a8acbf68f3940dce89a6 WatchSource:0}: Error finding container df2b256b863f2a77521ffb41b85876cf1da603e33af7a8acbf68f3940dce89a6: Status 404 returned error can't find the container with id df2b256b863f2a77521ffb41b85876cf1da603e33af7a8acbf68f3940dce89a6
Nov 04 03:48:52.499576 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.498603    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:48:52.502877 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:52.502857    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda021abf7_aecb_4044_8f56_86038fd77b5b.slice/crio-31e5a426a6112b4597dd55e9d617496d1de8aeb96583b41aaccc5f81167af172 WatchSource:0}: Error finding container 31e5a426a6112b4597dd55e9d617496d1de8aeb96583b41aaccc5f81167af172: Status 404 returned error can't find the container with id 31e5a426a6112b4597dd55e9d617496d1de8aeb96583b41aaccc5f81167af172
Nov 04 03:48:52.510371 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:52.510345    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4639182_9c75_4c83_81aa_e8e59d52fe6e.slice/crio-f276f8204b9a52b02e59f475a275f7e61cc344927976a6479b95da64e8e16292 WatchSource:0}: Error finding container f276f8204b9a52b02e59f475a275f7e61cc344927976a6479b95da64e8e16292: Status 404 returned error can't find the container with id f276f8204b9a52b02e59f475a275f7e61cc344927976a6479b95da64e8e16292
Nov 04 03:48:52.524605 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.524586    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:48:52.528468 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.528080    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:48:52.538132 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:52.538109    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6bb8b1d_e87b_4167_a84a_92e9815f2206.slice/crio-53db76be9492a37e78f045cf9979b5cb6912ead66e9b291d45d5a8fb63c4b6ea WatchSource:0}: Error finding container 53db76be9492a37e78f045cf9979b5cb6912ead66e9b291d45d5a8fb63c4b6ea: Status 404 returned error can't find the container with id 53db76be9492a37e78f045cf9979b5cb6912ead66e9b291d45d5a8fb63c4b6ea
Nov 04 03:48:52.540532 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.540230    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:48:52.552125 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:48:52.552103    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bb68137_3e53_42d0_a9fd_e6cddab3346c.slice/crio-7eb7017d429d5a601074116d7db76253931352236a7d83f7277bcc72e1a25745 WatchSource:0}: Error finding container 7eb7017d429d5a601074116d7db76253931352236a7d83f7277bcc72e1a25745: Status 404 returned error can't find the container with id 7eb7017d429d5a601074116d7db76253931352236a7d83f7277bcc72e1a25745
Nov 04 03:48:52.876667 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.876562    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:52.876667 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.876612    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:52.876916 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:52.876702    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:52.876916 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.876777    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:48:52.876916 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.876844    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:48:52.876916 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.876858    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs podName:37e230e6-a451-475a-8b79-9d000021644b nodeName:}" failed. No retries permitted until 2026-11-04 03:48:53.876837934 +0000 UTC m=+4.035892781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs") pod "network-metrics-daemon-hn8bx" (UID: "37e230e6-a451-475a-8b79-9d000021644b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:48:52.876916 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.876865    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:48:52.876916 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.876879    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ggjv6 for pod openshift-network-diagnostics/network-check-target-g97j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:48:52.876916 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.876880    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:48:52.877161 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.876930    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6 podName:4bbeb235-acfa-4f82-8edb-dc0727347f40 nodeName:}" failed. No retries permitted until 2026-11-04 03:48:53.876911879 +0000 UTC m=+4.035966740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggjv6" (UniqueName: "kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6") pod "network-check-target-g97j2" (UID: "4bbeb235-acfa-4f82-8edb-dc0727347f40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:48:52.877161 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:52.876962    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret podName:e38ea962-08ee-44b8-8872-feec06d36ada nodeName:}" failed. No retries permitted until 2026-11-04 03:48:53.876941014 +0000 UTC m=+4.035995859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret") pod "global-pull-secret-syncer-ksb9x" (UID: "e38ea962-08ee-44b8-8872-feec06d36ada") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:48:53.212963 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.212846    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:43:52 +0000 UTC" deadline="2028-04-29 08:18:34.339859928 +0000 UTC"
Nov 04 03:48:53.212963 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.212914    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13012h29m41.126950942s"
Nov 04 03:48:53.477762 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.477228    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:53.477762 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:53.477357    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:48:53.477994 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.477796    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:53.477994 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:53.477895    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:48:53.499116 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.499083    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5" event={"ID":"0bb68137-3e53-42d0-a9fd-e6cddab3346c","Type":"ContainerStarted","Data":"7eb7017d429d5a601074116d7db76253931352236a7d83f7277bcc72e1a25745"}
Nov 04 03:48:53.506949 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.506891    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vqkzw" event={"ID":"b9b6d0b0-68ba-4d89-9476-2727609a1264","Type":"ContainerStarted","Data":"9720f4fd3d9c516912ed18b22275262cd8dfc63cda1d7b06c7e23eb67e603b4f"}
Nov 04 03:48:53.514693 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.514608    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7jmdr" event={"ID":"d4639182-9c75-4c83-81aa-e8e59d52fe6e","Type":"ContainerStarted","Data":"f276f8204b9a52b02e59f475a275f7e61cc344927976a6479b95da64e8e16292"}
Nov 04 03:48:53.517929 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.517870    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wmvzs" event={"ID":"931e1236-fd6a-40f0-bd32-6e2f217eb5ff","Type":"ContainerStarted","Data":"594c966f892813491d1f7cc9e017421c2dc1870030368c603635501ced8aa703"}
Nov 04 03:48:53.522170 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.522040    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r" event={"ID":"86f60625-8953-4f46-aa16-cc1d9142c4df","Type":"ContainerStarted","Data":"36508a0c4387cbd148e95192f28fdfd39b34a9db94d58ac0e986eac0b68285cc"}
Nov 04 03:48:53.523725 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.523639    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-49hmt" event={"ID":"d6bb8b1d-e87b-4167-a84a-92e9815f2206","Type":"ContainerStarted","Data":"53db76be9492a37e78f045cf9979b5cb6912ead66e9b291d45d5a8fb63c4b6ea"}
Nov 04 03:48:53.525116 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.525091    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lw2sq" event={"ID":"a021abf7-aecb-4044-8f56-86038fd77b5b","Type":"ContainerStarted","Data":"31e5a426a6112b4597dd55e9d617496d1de8aeb96583b41aaccc5f81167af172"}
Nov 04 03:48:53.527184 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.527116    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-95jzm" event={"ID":"0982ff5a-da06-4180-9838-6bfc5f2deba0","Type":"ContainerStarted","Data":"df2b256b863f2a77521ffb41b85876cf1da603e33af7a8acbf68f3940dce89a6"}
Nov 04 03:48:53.887542 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.887465    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:53.887542 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.887515    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:53.887790 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:53.887548    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:53.887790 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:53.887686    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:48:53.887790 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:53.887744    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:48:53.887790 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:53.887766    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:48:53.887790 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:53.887778    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ggjv6 for pod openshift-network-diagnostics/network-check-target-g97j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:48:53.887790 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:53.887778    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:48:53.888106 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:53.887749    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs podName:37e230e6-a451-475a-8b79-9d000021644b nodeName:}" failed. No retries permitted until 2026-11-04 03:48:55.887729829 +0000 UTC m=+6.046784692 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs") pod "network-metrics-daemon-hn8bx" (UID: "37e230e6-a451-475a-8b79-9d000021644b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:48:53.888106 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:53.887825    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6 podName:4bbeb235-acfa-4f82-8edb-dc0727347f40 nodeName:}" failed. No retries permitted until 2026-11-04 03:48:55.887812933 +0000 UTC m=+6.046867774 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggjv6" (UniqueName: "kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6") pod "network-check-target-g97j2" (UID: "4bbeb235-acfa-4f82-8edb-dc0727347f40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:48:53.888106 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:53.887842    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret podName:e38ea962-08ee-44b8-8872-feec06d36ada nodeName:}" failed. No retries permitted until 2026-11-04 03:48:55.887832226 +0000 UTC m=+6.046887068 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret") pod "global-pull-secret-syncer-ksb9x" (UID: "e38ea962-08ee-44b8-8872-feec06d36ada") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:48:54.213182 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:54.213070    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:43:52 +0000 UTC" deadline="2028-05-31 13:08:05.666754762 +0000 UTC"
Nov 04 03:48:54.213182 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:54.213109    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13785h19m11.453649764s"
Nov 04 03:48:54.478002 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:54.477885    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:54.479179 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:54.478850    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:48:55.477115 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:55.476806    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:55.477591 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:55.477175    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:48:55.477591 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:55.476864    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:55.477743 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:55.477628    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:48:55.544850 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:55.544058    2573 generic.go:356] "Generic (PLEG): container finished" podID="fecc599dcc2092a8cb017231452a28ae" containerID="68d5ea0481c356ceb02c5cd997292eda75d9ce2fefcf56cf31cce9cb182ff64b" exitCode=0
Nov 04 03:48:55.544850 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:55.544151    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal" event={"ID":"fecc599dcc2092a8cb017231452a28ae","Type":"ContainerDied","Data":"68d5ea0481c356ceb02c5cd997292eda75d9ce2fefcf56cf31cce9cb182ff64b"}
Nov 04 03:48:55.549658 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:55.549254    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal" event={"ID":"f909644ddda3f20f4057bfd2523b2509","Type":"ContainerStarted","Data":"cc2a1f35b0cc4ae28fdff944c21ae5fcff47bb030d1020c10a80859ff05cf347"}
Nov 04 03:48:55.905865 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:55.905827    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:55.905983 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:55.905888    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:55.905983 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:55.905946    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:55.906098 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:55.906081    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:48:55.906161 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:55.906150    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs podName:37e230e6-a451-475a-8b79-9d000021644b nodeName:}" failed. No retries permitted until 2026-11-04 03:48:59.906130942 +0000 UTC m=+10.065185783 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs") pod "network-metrics-daemon-hn8bx" (UID: "37e230e6-a451-475a-8b79-9d000021644b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:48:55.906640 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:55.906623    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:48:55.906703 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:55.906650    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:48:55.906703 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:55.906663    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ggjv6 for pod openshift-network-diagnostics/network-check-target-g97j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:48:55.906788 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:55.906765    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6 podName:4bbeb235-acfa-4f82-8edb-dc0727347f40 nodeName:}" failed. No retries permitted until 2026-11-04 03:48:59.906744829 +0000 UTC m=+10.065799670 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggjv6" (UniqueName: "kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6") pod "network-check-target-g97j2" (UID: "4bbeb235-acfa-4f82-8edb-dc0727347f40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:48:55.906852 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:55.906839    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:48:55.906892 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:55.906883    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret podName:e38ea962-08ee-44b8-8872-feec06d36ada nodeName:}" failed. No retries permitted until 2026-11-04 03:48:59.906871012 +0000 UTC m=+10.065925859 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret") pod "global-pull-secret-syncer-ksb9x" (UID: "e38ea962-08ee-44b8-8872-feec06d36ada") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:48:56.477853 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:56.477819    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:56.478366 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:56.477974    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:48:56.558736 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:56.558489    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal" event={"ID":"fecc599dcc2092a8cb017231452a28ae","Type":"ContainerStarted","Data":"59f5307dfa2efa267a0eadf23992ad0689040a12657f649b458b84e292d9043a"}
Nov 04 03:48:56.568400 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:56.567251    2573 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="de2383377be564c5877aa2f22b05c45a958aca62d96d2c840e2dd8a8671a6068" exitCode=0
Nov 04 03:48:56.568400 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:56.568342    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vqkzw" event={"ID":"b9b6d0b0-68ba-4d89-9476-2727609a1264","Type":"ContainerDied","Data":"de2383377be564c5877aa2f22b05c45a958aca62d96d2c840e2dd8a8671a6068"}
Nov 04 03:48:56.586337 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:56.586271    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal" podStartSLOduration=4.586252844 podStartE2EDuration="4.586252844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:55.614716098 +0000 UTC m=+5.773770960" watchObservedRunningTime="2026-11-04 03:48:56.586252844 +0000 UTC m=+6.745307712"
Nov 04 03:48:56.586808 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:56.586765    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal" podStartSLOduration=4.586754813 podStartE2EDuration="4.586754813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:56.582724607 +0000 UTC m=+6.741779472" watchObservedRunningTime="2026-11-04 03:48:56.586754813 +0000 UTC m=+6.745809680"
Nov 04 03:48:57.478206 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:57.477545    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:57.478206 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:57.477554    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:57.478206 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:57.477787    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:48:57.478206 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:57.477856    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:48:57.575007 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:57.574970    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-49hmt" event={"ID":"d6bb8b1d-e87b-4167-a84a-92e9815f2206","Type":"ContainerStarted","Data":"c2368e1768730e5775fc0bdc98666ff7c45ebe35f336fa093ebada1b62fba2e1"}
Nov 04 03:48:58.480004 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:58.479367    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:58.480004 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:58.479499    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:48:58.582698 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:58.582672    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7jmdr" event={"ID":"d4639182-9c75-4c83-81aa-e8e59d52fe6e","Type":"ContainerStarted","Data":"242e05c8b668a4c7c286ddf7da39dd33469c429d4d54c59f86bce9363638473b"}
Nov 04 03:48:58.601286 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:58.600468    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-49hmt" podStartSLOduration=3.149835967 podStartE2EDuration="7.600449653s" totalImagesPullingTime="4.450613686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:57.597531216 +0000 UTC m=+7.756586100" watchObservedRunningTime="2026-11-04 03:48:58.600449653 +0000 UTC m=+8.759504517"
Nov 04 03:48:59.477454 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:59.477378    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:59.477605 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:59.477518    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:48:59.477839 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:59.477820    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:59.477915 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:59.477897    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:48:59.589819 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:59.589785    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r" event={"ID":"86f60625-8953-4f46-aa16-cc1d9142c4df","Type":"ContainerStarted","Data":"a5300d04b676047e81bf597b27b65e86ae55ad978cc74c19d6736869eb8d8129"}
Nov 04 03:48:59.941390 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:59.941313    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:48:59.941538 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:59.941407    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:48:59.941538 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:48:59.941486    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:48:59.941671 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:59.941660    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:48:59.941767 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:59.941752    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret podName:e38ea962-08ee-44b8-8872-feec06d36ada nodeName:}" failed. No retries permitted until 2026-11-04 03:49:07.941730536 +0000 UTC m=+18.100785392 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret") pod "global-pull-secret-syncer-ksb9x" (UID: "e38ea962-08ee-44b8-8872-feec06d36ada") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:48:59.942302 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:59.942257    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:48:59.942370 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:59.942356    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs podName:37e230e6-a451-475a-8b79-9d000021644b nodeName:}" failed. No retries permitted until 2026-11-04 03:49:07.942340284 +0000 UTC m=+18.101395136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs") pod "network-metrics-daemon-hn8bx" (UID: "37e230e6-a451-475a-8b79-9d000021644b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:48:59.942514 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:59.942499    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:48:59.942573 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:59.942520    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:48:59.942573 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:59.942539    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ggjv6 for pod openshift-network-diagnostics/network-check-target-g97j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:48:59.942683 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:48:59.942579    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6 podName:4bbeb235-acfa-4f82-8edb-dc0727347f40 nodeName:}" failed. No retries permitted until 2026-11-04 03:49:07.942566362 +0000 UTC m=+18.101621208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggjv6" (UniqueName: "kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6") pod "network-check-target-g97j2" (UID: "4bbeb235-acfa-4f82-8edb-dc0727347f40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:49:00.479845 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:00.479817    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:00.480000 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:00.479948    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:00.574732 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:00.574710    2573 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 03:49:00.594907 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:00.594869    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r" event={"ID":"86f60625-8953-4f46-aa16-cc1d9142c4df","Type":"ContainerStarted","Data":"ac9712f8b8524e67e55f459920a298bd1d7fa0493582f6272554f8a71524a96b"}
Nov 04 03:49:01.361637 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.361384    2573 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T03:49:00.574729277Z","UUID":"64ebae81-1e9e-4f67-b649-08487653afd0","Handler":null,"Name":"","Endpoint":""}
Nov 04 03:49:01.364028 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.364005    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 03:49:01.364146 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.364038    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 03:49:01.476983 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.476957    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:01.477116 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.477001    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:01.477169 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:01.477104    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:01.477454 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:01.477431    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:01.598419 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.598383    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5n7tp" event={"ID":"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e","Type":"ContainerStarted","Data":"377a90d3d50cdc5ba356cc36ca448821e609db2657876a2c8867824e1987f5a5"}
Nov 04 03:49:01.601940 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.601913    2573 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="d14b2490a3a540cc8f6442cd0065cbe85d316e9515816d05818332ab7fbea45b" exitCode=0
Nov 04 03:49:01.602106 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.601998    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vqkzw" event={"ID":"b9b6d0b0-68ba-4d89-9476-2727609a1264","Type":"ContainerDied","Data":"d14b2490a3a540cc8f6442cd0065cbe85d316e9515816d05818332ab7fbea45b"}
Nov 04 03:49:01.605854 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.605827    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lw2sq" event={"ID":"a021abf7-aecb-4044-8f56-86038fd77b5b","Type":"ContainerStarted","Data":"dccdd0ddbff92a1c98c0f0aae4bca0d2067115cdd23d4eb40f05b399e87c59fe"}
Nov 04 03:49:01.616503 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.616434    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-5n7tp" podStartSLOduration=3.289701337 podStartE2EDuration="11.616423506s" totalImagesPullingTime="8.326722169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:01.616017102 +0000 UTC m=+11.775071962" watchObservedRunningTime="2026-11-04 03:49:01.616423506 +0000 UTC m=+11.775478367"
Nov 04 03:49:01.616606 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.616584    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7jmdr" podStartSLOduration=5.114540301 podStartE2EDuration="10.616580159s" totalImagesPullingTime="5.502039858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:48:58.601263281 +0000 UTC m=+8.760318143" watchObservedRunningTime="2026-11-04 03:49:01.616580159 +0000 UTC m=+11.775635020"
Nov 04 03:49:01.683638 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:01.683567    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-lw2sq" podStartSLOduration=2.399907943 podStartE2EDuration="10.683550576s" totalImagesPullingTime="8.283642633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:01.658357922 +0000 UTC m=+11.817412782" watchObservedRunningTime="2026-11-04 03:49:01.683550576 +0000 UTC m=+11.842605438"
Nov 04 03:49:02.241199 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:02.241167    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:49:02.241897 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:02.241876    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:49:02.479808 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:02.479611    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:02.479808 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:02.479766    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:02.610802 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:02.610768    2573 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="e3f72fa14f8aa14a38b2b57190838561b37722bf38930a4fa2a9899d8330f709" exitCode=0
Nov 04 03:49:02.611271 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:02.610851    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vqkzw" event={"ID":"b9b6d0b0-68ba-4d89-9476-2727609a1264","Type":"ContainerDied","Data":"e3f72fa14f8aa14a38b2b57190838561b37722bf38930a4fa2a9899d8330f709"}
Nov 04 03:49:02.613396 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:02.613366    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wmvzs" event={"ID":"931e1236-fd6a-40f0-bd32-6e2f217eb5ff","Type":"ContainerStarted","Data":"de14681a585d9d3b96d5a76f9d1f308c94d9146e6049c5afa9cab64f3e408cde"}
Nov 04 03:49:02.616010 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:02.615986    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r" event={"ID":"86f60625-8953-4f46-aa16-cc1d9142c4df","Type":"ContainerStarted","Data":"49aa35c723d604644184b0eff97d5eae227b53a146caf84a9ff22d2f7ef6a08c"}
Nov 04 03:49:02.659292 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:02.659251    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r" podStartSLOduration=3.85865659 podStartE2EDuration="12.659235126s" totalImagesPullingTime="8.800578536s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:02.658755346 +0000 UTC m=+12.817810240" watchObservedRunningTime="2026-11-04 03:49:02.659235126 +0000 UTC m=+12.818289987"
Nov 04 03:49:02.675959 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:02.675921    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wmvzs" podStartSLOduration=4.113447047 podStartE2EDuration="12.67590462s" totalImagesPullingTime="8.562457573s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:02.675352981 +0000 UTC m=+12.834407843" watchObservedRunningTime="2026-11-04 03:49:02.67590462 +0000 UTC m=+12.834959485"
Nov 04 03:49:03.477084 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:03.477015    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:03.477084 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:03.477064    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:03.477272 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:03.477148    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:03.477677 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:03.477647    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:03.620082 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:03.620049    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-95jzm" event={"ID":"0982ff5a-da06-4180-9838-6bfc5f2deba0","Type":"ContainerStarted","Data":"df90f0896ddb9bec1b19ff5e7dd51e6b8c44ae350668a65e0354b2d90a3d751b"}
Nov 04 03:49:03.640816 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:03.640769    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-95jzm" podStartSLOduration=1.967018721 podStartE2EDuration="12.640752436s" totalImagesPullingTime="10.673733715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:03.63997239 +0000 UTC m=+13.799027255" watchObservedRunningTime="2026-11-04 03:49:03.640752436 +0000 UTC m=+13.799807299"
Nov 04 03:49:04.477316 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:04.477288    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:04.477490 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:04.477427    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:04.624719 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:04.624682    2573 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="ee49abd14e404ac2c8a70ed0ab9aba9a7e329418ed7fc78d2e3d5987dd402258" exitCode=0
Nov 04 03:49:04.625188 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:04.624768    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vqkzw" event={"ID":"b9b6d0b0-68ba-4d89-9476-2727609a1264","Type":"ContainerDied","Data":"ee49abd14e404ac2c8a70ed0ab9aba9a7e329418ed7fc78d2e3d5987dd402258"}
Nov 04 03:49:05.477916 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:05.477678    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:05.478091 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:05.477700    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:05.478091 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:05.478031    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:05.478208 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:05.478105    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:06.477105 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:06.477072    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:06.477543 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:06.477219    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:07.477063 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:07.477027    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:07.477063 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:07.477055    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:07.477540 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:07.477128    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:07.477540 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:07.477245    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:07.632936 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:07.632912    2573 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="168a7d05ba3a5119c9b280a16a367840072a4f038cf9742c3e75cbab63dc1baf" exitCode=0
Nov 04 03:49:07.633054 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:07.632948    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vqkzw" event={"ID":"b9b6d0b0-68ba-4d89-9476-2727609a1264","Type":"ContainerDied","Data":"168a7d05ba3a5119c9b280a16a367840072a4f038cf9742c3e75cbab63dc1baf"}
Nov 04 03:49:07.674550 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:07.674527    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:07.936978 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:07.936937    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:08.007459 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.007437    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:08.007567 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.007470    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:08.007567 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.007501    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:08.007567 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:08.007555    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:49:08.007726 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:08.007576    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:49:08.007726 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:08.007580    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:49:08.007726 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:08.007602    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ggjv6 for pod openshift-network-diagnostics/network-check-target-g97j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:49:08.007726 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:08.007637    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs podName:37e230e6-a451-475a-8b79-9d000021644b nodeName:}" failed. No retries permitted until 2026-11-04 03:49:24.007607712 +0000 UTC m=+34.166662564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs") pod "network-metrics-daemon-hn8bx" (UID: "37e230e6-a451-475a-8b79-9d000021644b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:49:08.007726 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:08.007576    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:49:08.007726 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:08.007664    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6 podName:4bbeb235-acfa-4f82-8edb-dc0727347f40 nodeName:}" failed. No retries permitted until 2026-11-04 03:49:24.007650072 +0000 UTC m=+34.166704927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggjv6" (UniqueName: "kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6") pod "network-check-target-g97j2" (UID: "4bbeb235-acfa-4f82-8edb-dc0727347f40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:49:08.007726 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:08.007683    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret podName:e38ea962-08ee-44b8-8872-feec06d36ada nodeName:}" failed. No retries permitted until 2026-11-04 03:49:24.007674366 +0000 UTC m=+34.166729205 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret") pod "global-pull-secret-syncer-ksb9x" (UID: "e38ea962-08ee-44b8-8872-feec06d36ada") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:49:08.319381 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.319173    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:08.477459 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.477435    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:08.477964 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:08.477530    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:08.637009 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.636981    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5" event={"ID":"0bb68137-3e53-42d0-a9fd-e6cddab3346c","Type":"ContainerStarted","Data":"15c63615964497d84feee50c8bd4ce1b5777b1fd9f7944f6399370deb496cdc3"}
Nov 04 03:49:08.637128 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.637046    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5" event={"ID":"0bb68137-3e53-42d0-a9fd-e6cddab3346c","Type":"ContainerStarted","Data":"06d01be00d272ef45053c9e178241883fd30963913995af9d34926edb64f4e2a"}
Nov 04 03:49:08.637128 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.637078    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5" event={"ID":"0bb68137-3e53-42d0-a9fd-e6cddab3346c","Type":"ContainerStarted","Data":"c58305513ec2e346392d5da027ed13f5411b3e31332e2ae32063ac87f9a36e91"}
Nov 04 03:49:08.637128 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.637105    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5" event={"ID":"0bb68137-3e53-42d0-a9fd-e6cddab3346c","Type":"ContainerStarted","Data":"c0c97bfb91e8b3cf8cf68b5309fe47106a48a80ef6119147a078b24ad2b99d23"}
Nov 04 03:49:08.637241 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.637139    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5" event={"ID":"0bb68137-3e53-42d0-a9fd-e6cddab3346c","Type":"ContainerStarted","Data":"59656de3310ad58f95c9a075569300d30ec02e3d86cbe73e99ca7ad4a8679dac"}
Nov 04 03:49:08.637241 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.637183    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5" event={"ID":"0bb68137-3e53-42d0-a9fd-e6cddab3346c","Type":"ContainerStarted","Data":"d86bce3a841160ee90083ad1b4372405fcc245707361ba06f5f7806e6d49e0f6"}
Nov 04 03:49:08.639742 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.639721    2573 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="1af84be52b05c51f2be520ae3d5d6d390ff568844a30210dd53ac49b79575c24" exitCode=0
Nov 04 03:49:08.639823 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:08.639754    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vqkzw" event={"ID":"b9b6d0b0-68ba-4d89-9476-2727609a1264","Type":"ContainerDied","Data":"1af84be52b05c51f2be520ae3d5d6d390ff568844a30210dd53ac49b79575c24"}
Nov 04 03:49:09.170007 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:09.169980    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:09.371199 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:09.371166    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:09.477270 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:09.477195    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:09.477270 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:09.477212    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:09.477437 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:09.477297    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:09.477437 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:09.477396    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:09.645188 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:09.645151    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vqkzw" event={"ID":"b9b6d0b0-68ba-4d89-9476-2727609a1264","Type":"ContainerStarted","Data":"230803801a31ecd14cb7a46ecb794c6c817d61569858ee10e4967a3d5f02c22d"}
Nov 04 03:49:09.672196 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:09.672156    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vqkzw" podStartSLOduration=7.139609316 podStartE2EDuration="18.672144462s" totalImagesPullingTime="11.532535146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:09.671924455 +0000 UTC m=+19.830979326" watchObservedRunningTime="2026-11-04 03:49:09.672144462 +0000 UTC m=+19.831199323"
Nov 04 03:49:10.479653 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:10.479612    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:10.480079 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:10.479721    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:10.650471 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:10.650438    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5" event={"ID":"0bb68137-3e53-42d0-a9fd-e6cddab3346c","Type":"ContainerStarted","Data":"feaf7a92b33087fe1b725a6e5666a3801b5c7b492dbfee35962f2bad5ce7e191"}
Nov 04 03:49:11.187361 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:11.187335    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:49:11.187525 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:11.187461    2573 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:49:11.187855 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:11.187841    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:49:11.364237 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:11.364204    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:11.477027 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:11.476973    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:11.477027 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:11.476986    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:11.477164 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:11.477053    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:11.477164 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:11.477157    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:11.571167 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:11.571141    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:12.007397 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:12.007366    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:12.277462 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:12.277386    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:12.477725 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:12.477565    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:12.477977 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:12.477831    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:12.514107 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:12.514076    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:12.657396 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:12.657363    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5" event={"ID":"0bb68137-3e53-42d0-a9fd-e6cddab3346c","Type":"ContainerStarted","Data":"dd52e825aea4cf2060e9a65fe035c7af351ff59417ac65a48b8d0dc034166c9a"}
Nov 04 03:49:12.657846 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:12.657615    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:49:12.672953 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:12.672934    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:49:12.693199 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:12.693162    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5" podStartSLOduration=6.288173504 podStartE2EDuration="21.693149981s" totalImagesPullingTime="15.404976477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:12.692682076 +0000 UTC m=+22.851736937" watchObservedRunningTime="2026-11-04 03:49:12.693149981 +0000 UTC m=+22.852204842"
Nov 04 03:49:12.721165 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:12.721143    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:13.410987 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:13.410960    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5n7tp_4cc23bbb-cc49-4e50-a94a-df074fcd4d3e/dns-node-resolver/0.log"
Nov 04 03:49:13.452016 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:13.451990    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:13.477376 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:13.477357    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:13.477487 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:13.477468    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:13.477544 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:13.477534    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:13.477641 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:13.477607    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:13.661238 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:13.661161    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:49:13.661956 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:13.661936    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:49:13.663502 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:13.663454    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:13.679555 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:13.679537    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:49:13.896557 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:13.896522    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:14.150829 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:14.150792    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:14.223967 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:14.223872    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ksb9x"]
Nov 04 03:49:14.224136 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:14.224121    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:14.224260 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:14.224239    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:14.225021 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:14.224993    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g97j2"]
Nov 04 03:49:14.225115 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:14.225085    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:14.225173 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:14.225158    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:14.227584 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:14.227563    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hn8bx"]
Nov 04 03:49:14.227688 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:14.227669    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:14.227759 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:14.227744    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:14.367751 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:14.367716    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:14.409474 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:14.409456    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7jmdr_d4639182-9c75-4c83-81aa-e8e59d52fe6e/node-ca/0.log"
Nov 04 03:49:14.572754 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:14.572696    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:15.477103 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:15.477076    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:15.477103 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:15.477091    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:15.477602 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:15.477115    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:15.477602 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:15.477187    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:15.477602 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:15.477252    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:15.477602 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:15.477371    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:15.490014 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:15.489992    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:15.886645 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:15.886608    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:16.091485 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:16.091454    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:16.298496 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:16.298434    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:16.499603 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:16.499576    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:16.708467 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:16.708439    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:17.477497 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:17.477426    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:17.477643 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:17.477519    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:17.477643 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:17.477527    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:17.477643 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:17.477596    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:17.477643 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:17.477607    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:17.477772 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:17.477718    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:17.497645 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:17.497609    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:17.705355 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:17.705328    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:17.911160 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:17.911129    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:18.117319 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:18.117286    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:18.328140 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:18.328107    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:18.684376 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:18.684348    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:18.897195 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:18.897169    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:19.099173 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:19.098985    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:19.476809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:19.476782    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:19.476809 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:19.476815    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:19.477126 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:19.476815    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:19.477126 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:19.476883    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:19.477126 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:19.476934    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:19.477126 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:19.477038    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:19.482962 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:19.482935    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:19.886459 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:19.886426    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:20.102297 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:20.102264    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:20.307773 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:20.307702    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:20.530532 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:20.530501    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:20.736871 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:20.736837    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:20.944980 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:20.944952    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:21.477373 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:21.477335    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:21.477373 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:21.477376    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:21.477808 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:21.477463    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:21.477808 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:21.477471    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:21.477808 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:21.477529    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:21.477808 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:21.477648    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:21.489112 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:21.489053    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:21.890361 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:21.890335    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:22.094283 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:22.094248    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:22.315405 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:22.315336    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:22.692361 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:22.692331    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:23.010883 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:23.010820    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:23.476976 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:23.476948    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:23.477148 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:23.476981    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:23.477148 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:23.477050    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:23.477148 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:23.477066    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:23.477148 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:23.477116    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:23.477274 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:23.477177    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:23.489692 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:23.489665    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:23.693001 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:23.692977    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:23.898537 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:23.898512    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:24.020014 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:24.019995    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:24.020114 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:24.020023    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:24.020114 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:24.020049    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:24.020224 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:24.020135    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:49:24.020224 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:24.020160    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 03:49:24.020224 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:24.020176    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs podName:37e230e6-a451-475a-8b79-9d000021644b nodeName:}" failed. No retries permitted until 2026-11-04 03:49:56.020164471 +0000 UTC m=+66.179219323 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs") pod "network-metrics-daemon-hn8bx" (UID: "37e230e6-a451-475a-8b79-9d000021644b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 03:49:24.020224 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:24.020204    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 03:49:24.020224 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:24.020214    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret podName:e38ea962-08ee-44b8-8872-feec06d36ada nodeName:}" failed. No retries permitted until 2026-11-04 03:49:56.020197979 +0000 UTC m=+66.179252834 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret") pod "global-pull-secret-syncer-ksb9x" (UID: "e38ea962-08ee-44b8-8872-feec06d36ada") : object "kube-system"/"original-pull-secret" not registered
Nov 04 03:49:24.020224 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:24.020222    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 03:49:24.020435 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:24.020233    2573 projected.go:196] Error preparing data for projected volume kube-api-access-ggjv6 for pod openshift-network-diagnostics/network-check-target-g97j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:49:24.020435 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:24.020271    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6 podName:4bbeb235-acfa-4f82-8edb-dc0727347f40 nodeName:}" failed. No retries permitted until 2026-11-04 03:49:56.020261976 +0000 UTC m=+66.179316815 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggjv6" (UniqueName: "kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6") pod "network-check-target-g97j2" (UID: "4bbeb235-acfa-4f82-8edb-dc0727347f40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 03:49:24.108505 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:24.108476    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:24.363382 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:24.363348    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:24.690151 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:24.690056    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:24.898963 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:24.898923    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:25.432381 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:25.432331    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:25.477571 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:25.477549    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:25.477698 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:25.477658    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:25.477698 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:25.477670    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:25.477782 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:25.477731    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:25.477782 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:25.477739    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:25.477844 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:25.477802    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:25.966452 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:25.966419    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:27.477060 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:27.477031    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:27.477060 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:27.477067    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:27.477445 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:27.477062    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:27.477445 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:27.477136    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:27.477445 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:27.477213    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:27.477445 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:27.477284    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:28.844564 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:28.844522    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:29.048300 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:29.048270    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:29.476799 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:29.476776    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:29.476953 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:29.476867    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:29.476953 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:29.476871    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:29.476953 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:29.476890    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:29.477050 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:29.476956    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:29.477050 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:29.477012    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:31.477349 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:31.477318    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:31.477349 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:31.477350    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:31.477349 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:31.477347    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:31.477922 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:31.477437    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:31.477922 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:31.477484    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:31.477922 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:31.477535    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:33.477441 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:33.477409    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:33.477880 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:33.477409    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:33.477880 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:33.477531    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:33.477880 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:33.477409    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:33.477880 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:33.477578    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:33.477880 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:33.477677    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:35.477466 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:35.477427    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:35.477466 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:35.477473    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:35.477961 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:35.477500    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:35.477961 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:35.477592    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g97j2" podUID="4bbeb235-acfa-4f82-8edb-dc0727347f40"
Nov 04 03:49:35.477961 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:35.477666    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ksb9x" podUID="e38ea962-08ee-44b8-8872-feec06d36ada"
Nov 04 03:49:35.477961 ip-10-0-129-253 kubenswrapper[2573]: E1104 03:49:35.477762    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hn8bx" podUID="37e230e6-a451-475a-8b79-9d000021644b"
Nov 04 03:49:35.531466 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:35.531417    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:35.738688 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:35.738588    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:36.697540 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.697514    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeReady"
Nov 04 03:49:36.697941 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.697658    2573 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 03:49:36.750528 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.750500    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lcn4v"]
Nov 04 03:49:36.752146 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.752131    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:36.755878 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.755858    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s4h9x"]
Nov 04 03:49:36.757156 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.757135    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 03:49:36.757265 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.757174    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ccqtd\""
Nov 04 03:49:36.757265 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.757226    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:49:36.759858 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.759843    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 03:49:36.766217 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.766199    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lcn4v"]
Nov 04 03:49:36.769457 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.769431    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s4h9x"]
Nov 04 03:49:36.776825 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.776808    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qkmp2"]
Nov 04 03:49:36.778291 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.778277    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:36.794729 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.794712    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qkmp2"]
Nov 04 03:49:36.857457 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.857438    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 03:49:36.857604 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.857589    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 03:49:36.857699 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.857681    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-49k7r\""
Nov 04 03:49:36.857699 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.857694    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n475p\""
Nov 04 03:49:36.857783 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.857695    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 03:49:36.857817 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.857782    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 03:49:36.857912 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.857898    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 03:49:36.858421 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.858410    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 03:49:36.858463 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.858438    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 03:49:36.916211 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916192    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9062ed54-5f71-4f96-8274-20ab5a38a9a0-data-volume\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:36.916265 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916216    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68df7a49-2cdd-46c3-8730-9447708be784-cert\") pod \"ingress-canary-s4h9x\" (UID: \"68df7a49-2cdd-46c3-8730-9447708be784\") " pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:49:36.916265 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916251    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9062ed54-5f71-4f96-8274-20ab5a38a9a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:36.916358 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916319    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jzxp\" (UniqueName: \"kubernetes.io/projected/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-api-access-7jzxp\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:36.916410 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916373    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f25400e-aea9-43e3-b17f-0a68a76a4632-config-volume\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:36.916410 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916399    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f25400e-aea9-43e3-b17f-0a68a76a4632-tmp-dir\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:36.916508 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916448    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:36.916508 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916477    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt9sb\" (UniqueName: \"kubernetes.io/projected/68df7a49-2cdd-46c3-8730-9447708be784-kube-api-access-nt9sb\") pod \"ingress-canary-s4h9x\" (UID: \"68df7a49-2cdd-46c3-8730-9447708be784\") " pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:49:36.916588 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916558    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsvkq\" (UniqueName: \"kubernetes.io/projected/4f25400e-aea9-43e3-b17f-0a68a76a4632-kube-api-access-qsvkq\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:36.916588 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916582    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9062ed54-5f71-4f96-8274-20ab5a38a9a0-crio-socket\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:36.916691 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:36.916608    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f25400e-aea9-43e3-b17f-0a68a76a4632-metrics-tls\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:37.017554 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017494    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9062ed54-5f71-4f96-8274-20ab5a38a9a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.017554 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017527    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jzxp\" (UniqueName: \"kubernetes.io/projected/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-api-access-7jzxp\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.017720 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017557    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f25400e-aea9-43e3-b17f-0a68a76a4632-config-volume\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:37.017720 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017578    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f25400e-aea9-43e3-b17f-0a68a76a4632-tmp-dir\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:37.017720 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017636    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.017720 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017665    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt9sb\" (UniqueName: \"kubernetes.io/projected/68df7a49-2cdd-46c3-8730-9447708be784-kube-api-access-nt9sb\") pod \"ingress-canary-s4h9x\" (UID: \"68df7a49-2cdd-46c3-8730-9447708be784\") " pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:49:37.017913 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017772    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsvkq\" (UniqueName: \"kubernetes.io/projected/4f25400e-aea9-43e3-b17f-0a68a76a4632-kube-api-access-qsvkq\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:37.017913 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017809    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9062ed54-5f71-4f96-8274-20ab5a38a9a0-crio-socket\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.017913 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017846    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f25400e-aea9-43e3-b17f-0a68a76a4632-metrics-tls\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:37.017913 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017875    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9062ed54-5f71-4f96-8274-20ab5a38a9a0-data-volume\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.017913 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017894    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9062ed54-5f71-4f96-8274-20ab5a38a9a0-crio-socket\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.018442 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.018419    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f25400e-aea9-43e3-b17f-0a68a76a4632-config-volume\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:37.018515 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.018435    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9062ed54-5f71-4f96-8274-20ab5a38a9a0-data-volume\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.018569 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.017922    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68df7a49-2cdd-46c3-8730-9447708be784-cert\") pod \"ingress-canary-s4h9x\" (UID: \"68df7a49-2cdd-46c3-8730-9447708be784\") " pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:49:37.018718 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.018694    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f25400e-aea9-43e3-b17f-0a68a76a4632-tmp-dir\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:37.018955 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.018931    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.022865 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.022831    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9062ed54-5f71-4f96-8274-20ab5a38a9a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.022948 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.022886    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f25400e-aea9-43e3-b17f-0a68a76a4632-metrics-tls\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:37.024041 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.024022    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68df7a49-2cdd-46c3-8730-9447708be784-cert\") pod \"ingress-canary-s4h9x\" (UID: \"68df7a49-2cdd-46c3-8730-9447708be784\") " pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:49:37.030414 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.030393    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsvkq\" (UniqueName: \"kubernetes.io/projected/4f25400e-aea9-43e3-b17f-0a68a76a4632-kube-api-access-qsvkq\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:37.030707 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.030686    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jzxp\" (UniqueName: \"kubernetes.io/projected/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-api-access-7jzxp\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.030793 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.030738    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt9sb\" (UniqueName: \"kubernetes.io/projected/68df7a49-2cdd-46c3-8730-9447708be784-kube-api-access-nt9sb\") pod \"ingress-canary-s4h9x\" (UID: \"68df7a49-2cdd-46c3-8730-9447708be784\") " pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:49:37.064462 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.064433    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:37.068575 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.068560    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:49:37.089343 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.089310    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:49:37.384876 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.384847    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s4h9x"]
Nov 04 03:49:37.385611 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.385543    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lcn4v"]
Nov 04 03:49:37.386469 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:37.386441    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68df7a49_2cdd_46c3_8730_9447708be784.slice/crio-d0d74ca4f35221090e986f4163b534d866537d1f570aefd78de83725ee788a09 WatchSource:0}: Error finding container d0d74ca4f35221090e986f4163b534d866537d1f570aefd78de83725ee788a09: Status 404 returned error can't find the container with id d0d74ca4f35221090e986f4163b534d866537d1f570aefd78de83725ee788a09
Nov 04 03:49:37.387851 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:37.387822    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f25400e_aea9_43e3_b17f_0a68a76a4632.slice/crio-3dbc6454498ef5643c21f406401cd53cea0aa791e9aad6e6d5a996202ed5d896 WatchSource:0}: Error finding container 3dbc6454498ef5643c21f406401cd53cea0aa791e9aad6e6d5a996202ed5d896: Status 404 returned error can't find the container with id 3dbc6454498ef5643c21f406401cd53cea0aa791e9aad6e6d5a996202ed5d896
Nov 04 03:49:37.398269 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.398248    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qkmp2"]
Nov 04 03:49:37.400647 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:37.400614    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9062ed54_5f71_4f96_8274_20ab5a38a9a0.slice/crio-2fc3d49c43c6a5cdaf40e75b563deb9a951dd7032324a2304bc5dcb4f3988fc8 WatchSource:0}: Error finding container 2fc3d49c43c6a5cdaf40e75b563deb9a951dd7032324a2304bc5dcb4f3988fc8: Status 404 returned error can't find the container with id 2fc3d49c43c6a5cdaf40e75b563deb9a951dd7032324a2304bc5dcb4f3988fc8
Nov 04 03:49:37.476781 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.476759    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:37.476881 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.476819    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:37.476881 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.476843    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:37.480840 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.480820    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 03:49:37.480840 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.480835    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 03:49:37.480960 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.480867    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-49g8t\""
Nov 04 03:49:37.480960 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.480907    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-p2j7s\""
Nov 04 03:49:37.482148 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.482125    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 03:49:37.482148 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.482130    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 03:49:37.482310 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.482296    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfbg\""
Nov 04 03:49:37.709752 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.709677    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s4h9x" event={"ID":"68df7a49-2cdd-46c3-8730-9447708be784","Type":"ContainerStarted","Data":"d0d74ca4f35221090e986f4163b534d866537d1f570aefd78de83725ee788a09"}
Nov 04 03:49:37.711007 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.710987    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qkmp2" event={"ID":"9062ed54-5f71-4f96-8274-20ab5a38a9a0","Type":"ContainerStarted","Data":"f87b736d2a7e0bda21d653d5e8e8f18a413b9048150463614bd1d95196ea13e5"}
Nov 04 03:49:37.711097 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.711028    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qkmp2" event={"ID":"9062ed54-5f71-4f96-8274-20ab5a38a9a0","Type":"ContainerStarted","Data":"2fc3d49c43c6a5cdaf40e75b563deb9a951dd7032324a2304bc5dcb4f3988fc8"}
Nov 04 03:49:37.712060 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.712028    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lcn4v" event={"ID":"4f25400e-aea9-43e3-b17f-0a68a76a4632","Type":"ContainerStarted","Data":"3dbc6454498ef5643c21f406401cd53cea0aa791e9aad6e6d5a996202ed5d896"}
Nov 04 03:49:37.855417 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:37.855396    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5n7tp_4cc23bbb-cc49-4e50-a94a-df074fcd4d3e/dns-node-resolver/0.log"
Nov 04 03:49:38.716275 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:38.716245    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qkmp2" event={"ID":"9062ed54-5f71-4f96-8274-20ab5a38a9a0","Type":"ContainerStarted","Data":"316f93f824684a666e52057d8eafe7069d56d8b3df1199b89cfc5dfc2cdecf1d"}
Nov 04 03:49:38.806776 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:38.806568    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:39.017802 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:39.017581    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:39.057421 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:39.057391    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7jmdr_d4639182-9c75-4c83-81aa-e8e59d52fe6e/node-ca/0.log"
Nov 04 03:49:39.721794 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:39.721696    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lcn4v" event={"ID":"4f25400e-aea9-43e3-b17f-0a68a76a4632","Type":"ContainerStarted","Data":"61e3b994da862c2d4b9818f9ed6e30ef4fda3f884d3f40977d9260e35fc8cf1e"}
Nov 04 03:49:39.721794 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:39.721754    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lcn4v" event={"ID":"4f25400e-aea9-43e3-b17f-0a68a76a4632","Type":"ContainerStarted","Data":"8b9d44a916cfa953097e39deb70e2190a6c772cce5fa47c01642b8287d50aa6f"}
Nov 04 03:49:39.722212 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:39.721838    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:39.746342 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:39.746261    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-lcn4v" podStartSLOduration=1.660321379 podStartE2EDuration="3.746249064s" totalImagesPullingTime="2.085927685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:39.746179297 +0000 UTC m=+49.905234159" watchObservedRunningTime="2026-11-04 03:49:39.746249064 +0000 UTC m=+49.905303924"
Nov 04 03:49:39.802895 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:39.802864    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:40.007873 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:40.007831    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:40.725838 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:40.725744    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qkmp2" event={"ID":"9062ed54-5f71-4f96-8274-20ab5a38a9a0","Type":"ContainerStarted","Data":"76f37c7dc68e73e31048dcdff1e9d2cac2b21d8f42e68c362c05eaad2426cc3c"}
Nov 04 03:49:40.727177 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:40.727147    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s4h9x" event={"ID":"68df7a49-2cdd-46c3-8730-9447708be784","Type":"ContainerStarted","Data":"55708a7394e0052cc2bc38ccc36a295f9552d2e205cbfeda707a798a84fbcc13"}
Nov 04 03:49:40.747881 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:40.747836    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qkmp2" podStartSLOduration=2.301969905 podStartE2EDuration="4.747824646s" totalImagesPullingTime="2.445854741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 03:49:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:40.747008663 +0000 UTC m=+50.906063523" watchObservedRunningTime="2026-11-04 03:49:40.747824646 +0000 UTC m=+50.906879506"
Nov 04 03:49:40.764587 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:40.764548    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s4h9x" podStartSLOduration=1.71709358 podStartE2EDuration="4.764537263s" totalImagesPullingTime="3.047443683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:40.763918927 +0000 UTC m=+50.922973788" watchObservedRunningTime="2026-11-04 03:49:40.764537263 +0000 UTC m=+50.923592125"
Nov 04 03:49:43.262096 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:43.262053    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:43.672366 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:43.672333    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:43.878124 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:43.878081    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:44.985889 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:44.985850    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:45.132803 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.132774    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"]
Nov 04 03:49:45.134813 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.134798    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.155806 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.155784    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nxhrm"]
Nov 04 03:49:45.156099 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.156075    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 03:49:45.156099 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.156092    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 03:49:45.156316 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.156299    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 04 03:49:45.157593 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.157575    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 04 03:49:45.157720 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.157611    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 03:49:45.157720 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.157612    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fg6jl\""
Nov 04 03:49:45.157720 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.157643    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.159946 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.159928    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"]
Nov 04 03:49:45.172833 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.172811    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"]
Nov 04 03:49:45.174551 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.174536    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.175291 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175270    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-root\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.175383 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175321    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.175383 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175344    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.175383 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175364    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv5wd\" (UniqueName: \"kubernetes.io/projected/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-kube-api-access-hv5wd\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.175383 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175380    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-textfile\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.175610 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175394    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.175610 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175440    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.175610 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175528    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqkww\" (UniqueName: \"kubernetes.io/projected/74f52bd8-5314-4ba0-8834-81e86a91c1f6-kube-api-access-sqkww\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.175610 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175551    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.175610 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175584    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-sys\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.175874 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175613    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/aae100a6-d602-4c8b-b301-2f7a7772fe77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.175874 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175692    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.175874 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175736    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.175874 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175754    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-wtmp\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.175874 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175773    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plbs2\" (UniqueName: \"kubernetes.io/projected/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-api-access-plbs2\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.175874 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175803    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-metrics-client-ca\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.175874 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175852    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.176209 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175903    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.176209 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.175927    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-tls\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.194277 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.194257    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"]
Nov 04 03:49:45.229106 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.229051    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 03:49:45.229198 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.229157    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 03:49:45.229308 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.229292    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 03:49:45.229343 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.229296    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rh4kv\""
Nov 04 03:49:45.229343 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.229321    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rjs4c\""
Nov 04 03:49:45.229343 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.229328    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 03:49:45.232953 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.232934    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:45.254807 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.254749    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 03:49:45.254972 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.254958    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 03:49:45.276382 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276359    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.276460 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276384    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.276460 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276405    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv5wd\" (UniqueName: \"kubernetes.io/projected/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-kube-api-access-hv5wd\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.276460 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276420    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-textfile\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.276460 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276437    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.276658 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276460    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.276658 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276519    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqkww\" (UniqueName: \"kubernetes.io/projected/74f52bd8-5314-4ba0-8834-81e86a91c1f6-kube-api-access-sqkww\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.276658 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276548    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.276658 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276569    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-sys\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.276658 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276594    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/aae100a6-d602-4c8b-b301-2f7a7772fe77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.276658 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276651    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.276943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276702    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.276943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276728    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-wtmp\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.276943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276761    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plbs2\" (UniqueName: \"kubernetes.io/projected/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-api-access-plbs2\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.276943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276787    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-metrics-client-ca\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.276943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276816    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.276943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276828    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-textfile\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.276943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276844    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.276943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276880    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-tls\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.276943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276915    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-root\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.277386 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.276987    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-root\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.277386 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.277299    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.277386 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.277320    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.277386 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.277362    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-sys\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.277386 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.277297    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.277641 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.277602    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-wtmp\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.277867 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.277841    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-metrics-client-ca\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.277973 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.277950    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/aae100a6-d602-4c8b-b301-2f7a7772fe77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.278310 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.278284    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.279420 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.279397    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.279532 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.279438    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-tls\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.279630 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.279599    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.279995 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.279968    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.280078 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.280021    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.280309 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.280286    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.301731 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.301702    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqkww\" (UniqueName: \"kubernetes.io/projected/74f52bd8-5314-4ba0-8834-81e86a91c1f6-kube-api-access-sqkww\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.301831 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.301814    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv5wd\" (UniqueName: \"kubernetes.io/projected/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-kube-api-access-hv5wd\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.307148 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.307128    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plbs2\" (UniqueName: \"kubernetes.io/projected/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-api-access-plbs2\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.440274 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.440230    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:45.446076 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.446051    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:49:45.469991 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.469964    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:49:45.480123 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:45.480096    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f52bd8_5314_4ba0_8834_81e86a91c1f6.slice/crio-786aaf26a4bf4d0051e0e886149fd2053217ddfa20d268c7d55c33fd43434cd8 WatchSource:0}: Error finding container 786aaf26a4bf4d0051e0e886149fd2053217ddfa20d268c7d55c33fd43434cd8: Status 404 returned error can't find the container with id 786aaf26a4bf4d0051e0e886149fd2053217ddfa20d268c7d55c33fd43434cd8
Nov 04 03:49:45.486042 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.486013    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:49:45.651211 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.651181    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:45.675126 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.675100    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:49:45.723201 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.723172    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"]
Nov 04 03:49:45.725941 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:45.725915    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf14b5_d08b_4a4f_af25_2e24d53fdf4d.slice/crio-67f0de77c5a45d70de89ea8b34a7ba039802bafdab6a37f94a42a3b4cbc81291 WatchSource:0}: Error finding container 67f0de77c5a45d70de89ea8b34a7ba039802bafdab6a37f94a42a3b4cbc81291: Status 404 returned error can't find the container with id 67f0de77c5a45d70de89ea8b34a7ba039802bafdab6a37f94a42a3b4cbc81291
Nov 04 03:49:45.739747 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.739715    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxhrm" event={"ID":"74f52bd8-5314-4ba0-8834-81e86a91c1f6","Type":"ContainerStarted","Data":"786aaf26a4bf4d0051e0e886149fd2053217ddfa20d268c7d55c33fd43434cd8"}
Nov 04 03:49:45.740988 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.740960    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl" event={"ID":"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d","Type":"ContainerStarted","Data":"67f0de77c5a45d70de89ea8b34a7ba039802bafdab6a37f94a42a3b4cbc81291"}
Nov 04 03:49:45.764285 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.764252    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"]
Nov 04 03:49:45.767271 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:45.767247    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae100a6_d602_4c8b_b301_2f7a7772fe77.slice/crio-12f7f6d603150db4d1f8d7d6504361a27aa4ec238a45f6fede6d616e652ee6ee WatchSource:0}: Error finding container 12f7f6d603150db4d1f8d7d6504361a27aa4ec238a45f6fede6d616e652ee6ee: Status 404 returned error can't find the container with id 12f7f6d603150db4d1f8d7d6504361a27aa4ec238a45f6fede6d616e652ee6ee
Nov 04 03:49:45.855659 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:45.855614    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:46.262367 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.262334    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:49:46.265116 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.265089    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.270479 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.270452    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 03:49:46.270723 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.270703    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 03:49:46.272097 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.271953    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 03:49:46.272097 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.271973    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5cbvs\""
Nov 04 03:49:46.272097 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.272059    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 03:49:46.272097 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.272067    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 03:49:46.272097 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.272071    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 03:49:46.272332 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.272148    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 03:49:46.272499 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.272477    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 03:49:46.273398 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.273333    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 03:49:46.284803 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.284783    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:49:46.286148 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286127    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286242 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286157    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286242 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286173    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286242 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286209    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286395 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286263    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p84j\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-kube-api-access-4p84j\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286395 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286349    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286490 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286387    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286490 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286468    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286610 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286590    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286687 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286653    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-volume\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286687 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286677    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-web-config\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286765 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286734    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.286805 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.286773    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-out\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.298490 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.298470    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:46.387827 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.387795    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.387939 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.387836    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p84j\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-kube-api-access-4p84j\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.387939 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.387864    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.387939 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.387881    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.387939 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.387898    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.388142 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.387946    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.388142 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.387970    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-volume\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.388142 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.387984    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-web-config\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.388142 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.388036    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.388142 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.388062    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-out\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.388142 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.388104    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.388142 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.388133    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.388386 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.388159    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.388778 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.388755    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.389887 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.389828    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.390301 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.388610    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.392721 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.392697    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.399690 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.399171    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-out\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.399690 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.399247    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.399690 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.399375    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.399938 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.399695    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.399938 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.399803    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-web-config\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.400109 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.399988    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.401371 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.401345    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-volume\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.404109 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.404086    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p84j\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-kube-api-access-4p84j\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.405166 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.405146    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.524419 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.524333    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:46.583076 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.583042    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:49:46.746937 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.746891    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc" event={"ID":"aae100a6-d602-4c8b-b301-2f7a7772fe77","Type":"ContainerStarted","Data":"12f7f6d603150db4d1f8d7d6504361a27aa4ec238a45f6fede6d616e652ee6ee"}
Nov 04 03:49:46.750684 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.750654    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl" event={"ID":"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d","Type":"ContainerStarted","Data":"5df72c17bf066f00792f67f96252defa72e46ab0a813be4628a45cde42fd4348"}
Nov 04 03:49:46.750819 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.750709    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl" event={"ID":"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d","Type":"ContainerStarted","Data":"90ffd29dbc54aaeab4dfa86fa21b1d518f3ae802ed029f7a7d35e6d8440e1b09"}
Nov 04 03:49:46.880153 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:46.880129    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:49:46.891569 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:46.891537    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1b8f136_f4a2_4c04_a35a_e60e2788d381.slice/crio-e433bbe80c3f57d3441236b9113fdbe7b990e68833e7f23814978187501c1f45 WatchSource:0}: Error finding container e433bbe80c3f57d3441236b9113fdbe7b990e68833e7f23814978187501c1f45: Status 404 returned error can't find the container with id e433bbe80c3f57d3441236b9113fdbe7b990e68833e7f23814978187501c1f45
Nov 04 03:49:47.759192 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:47.759114    2573 generic.go:356] "Generic (PLEG): container finished" podID="74f52bd8-5314-4ba0-8834-81e86a91c1f6" containerID="5cb364e40c845be6c7bfc2c2288189afdb15f8c0d0fbb6701b53e0eac49b6108" exitCode=0
Nov 04 03:49:47.759597 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:47.759204    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxhrm" event={"ID":"74f52bd8-5314-4ba0-8834-81e86a91c1f6","Type":"ContainerDied","Data":"5cb364e40c845be6c7bfc2c2288189afdb15f8c0d0fbb6701b53e0eac49b6108"}
Nov 04 03:49:47.762518 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:47.762453    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl" event={"ID":"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d","Type":"ContainerStarted","Data":"703c14bea3626dd006ecf6e3b697ad32bdfe7ac850fb006d7e760889f0b84e8c"}
Nov 04 03:49:47.764206 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:47.764179    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerStarted","Data":"e433bbe80c3f57d3441236b9113fdbe7b990e68833e7f23814978187501c1f45"}
Nov 04 03:49:47.767492 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:47.767466    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc" event={"ID":"aae100a6-d602-4c8b-b301-2f7a7772fe77","Type":"ContainerStarted","Data":"e6a76f732655284ecd8d133e14b2ccd2a45386bb4dac0f178b31979e466f9362"}
Nov 04 03:49:47.767612 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:47.767514    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc" event={"ID":"aae100a6-d602-4c8b-b301-2f7a7772fe77","Type":"ContainerStarted","Data":"b0b940bcd2c4a4587f54075f1b2c14c0fd6375a2f88d7eb61fad8c02a1cde9b6"}
Nov 04 03:49:47.767612 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:47.767541    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc" event={"ID":"aae100a6-d602-4c8b-b301-2f7a7772fe77","Type":"ContainerStarted","Data":"f2bbcee2182bfda2b8dd513d052ac406e2e34a3d7f920d344d4e90f5cda5d2e4"}
Nov 04 03:49:47.807135 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:47.807088    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc" podStartSLOduration=1.117117932 podStartE2EDuration="2.807074005s" totalImagesPullingTime="1.689956073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:47.806683951 +0000 UTC m=+57.965738813" watchObservedRunningTime="2026-11-04 03:49:47.807074005 +0000 UTC m=+57.966128867"
Nov 04 03:49:47.830450 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:47.830410    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl" podStartSLOduration=1.745359296 podStartE2EDuration="2.830398577s" totalImagesPullingTime="1.085039281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:47.829456648 +0000 UTC m=+57.988511510" watchObservedRunningTime="2026-11-04 03:49:47.830398577 +0000 UTC m=+57.989453437"
Nov 04 03:49:48.295337 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:48.295268    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:48.491280 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:48.491243    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:48.772336 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:48.772307    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxhrm" event={"ID":"74f52bd8-5314-4ba0-8834-81e86a91c1f6","Type":"ContainerStarted","Data":"35ee65a7e621f709ad38a2f617652c6003add267d684ebe28c0e0c64041d087d"}
Nov 04 03:49:48.772740 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:48.772358    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxhrm" event={"ID":"74f52bd8-5314-4ba0-8834-81e86a91c1f6","Type":"ContainerStarted","Data":"3aab91c63c801ccea54d03cefa764cb1f80b280183aa90a2a655c79959a5c689"}
Nov 04 03:49:48.773882 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:48.773860    2573 generic.go:356] "Generic (PLEG): container finished" podID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerID="0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103" exitCode=0
Nov 04 03:49:48.773964 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:48.773939    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerDied","Data":"0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103"}
Nov 04 03:49:48.797097 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:48.797062    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nxhrm" podStartSLOduration=2.340363789 podStartE2EDuration="3.797048306s" totalImagesPullingTime="1.380435026s" totalInitContainerRuntime="76.249491ms" isStatefulPod=true podCreationTimestamp="2026-11-04 03:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:48.79504403 +0000 UTC m=+58.954098881" watchObservedRunningTime="2026-11-04 03:49:48.797048306 +0000 UTC m=+58.956103190"
Nov 04 03:49:48.951275 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:48.951249    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:49.154707 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.154681    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:49.689530 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.689500    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5558d76459-nzh9f"]
Nov 04 03:49:49.692478 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.692460    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.700681 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.700660    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 03:49:49.700788 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.700684    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 04 03:49:49.704948 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.704924    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5558d76459-nzh9f"]
Nov 04 03:49:49.716136 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.716114    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-client-certs\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.716243 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.716143    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-tls\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.716243 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.716163    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-client-ca-bundle\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.716375 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.716357    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9b84a44a-cb54-47a0-834b-d5794c8d59ad-audit-log\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.716406 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.716384    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-metrics-server-audit-profiles\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.716452 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.716401    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2v7r\" (UniqueName: \"kubernetes.io/projected/9b84a44a-cb54-47a0-834b-d5794c8d59ad-kube-api-access-n2v7r\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.716498 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.716483    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.725721 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.725698    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 04 03:49:49.725867 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.725848    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 04 03:49:49.725867 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.725848    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mxr5f\""
Nov 04 03:49:49.726026 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.725866    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-74vjr0sga3qj\""
Nov 04 03:49:49.729578 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.729560    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:49:49.817003 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.816977    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9b84a44a-cb54-47a0-834b-d5794c8d59ad-audit-log\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.817003 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.817004    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-metrics-server-audit-profiles\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.817500 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.817023    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2v7r\" (UniqueName: \"kubernetes.io/projected/9b84a44a-cb54-47a0-834b-d5794c8d59ad-kube-api-access-n2v7r\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.817500 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.817107    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.817500 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.817142    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-client-certs\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.817500 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.817166    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-tls\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.817500 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.817196    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-client-ca-bundle\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.817500 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.817379    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9b84a44a-cb54-47a0-834b-d5794c8d59ad-audit-log\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.818214 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.818184    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.818318 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.818265    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-metrics-server-audit-profiles\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.819773 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.819748    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-client-ca-bundle\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.819873 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.819813    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-tls\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.819873 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.819841    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-client-certs\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:49.830806 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:49.830781    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2v7r\" (UniqueName: \"kubernetes.io/projected/9b84a44a-cb54-47a0-834b-d5794c8d59ad-kube-api-access-n2v7r\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:50.003851 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.003777    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:49:50.023921 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.023879    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:50.235022 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.234988    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:50.283519 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.283456    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5558d76459-nzh9f"]
Nov 04 03:49:50.286205 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:50.286172    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b84a44a_cb54_47a0_834b_d5794c8d59ad.slice/crio-70d4859ab57d2dc451d265bca42cee959115e784869cfd5dd6b09473a5bdca57 WatchSource:0}: Error finding container 70d4859ab57d2dc451d265bca42cee959115e784869cfd5dd6b09473a5bdca57: Status 404 returned error can't find the container with id 70d4859ab57d2dc451d265bca42cee959115e784869cfd5dd6b09473a5bdca57
Nov 04 03:49:50.384176 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.384144    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"]
Nov 04 03:49:50.388453 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.388434    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.392369 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.392338    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 04 03:49:50.392457 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.392388    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 04 03:49:50.392457 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.392392    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 04 03:49:50.392457 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.392415    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 04 03:49:50.392457 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.392450    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-p8xwh\""
Nov 04 03:49:50.401912 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.401890    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 04 03:49:50.402518 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.402498    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"]
Nov 04 03:49:50.406041 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.406020    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 04 03:49:50.422371 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.422345    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.422461 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.422386    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmlcb\" (UniqueName: \"kubernetes.io/projected/bd169cd6-f080-4fc4-82b4-cdb781a09640-kube-api-access-fmlcb\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.422461 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.422455    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-metrics-client-ca\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.422530 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.422503    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-federate-client-tls\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.422530 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.422523    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-serving-certs-ca-bundle\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.422603 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.422543    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-client-tls\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.422603 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.422585    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.422699 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.422604    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.523919 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.523887    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.524066 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.523931    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.524066 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.524002    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.524066 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.524054    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmlcb\" (UniqueName: \"kubernetes.io/projected/bd169cd6-f080-4fc4-82b4-cdb781a09640-kube-api-access-fmlcb\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.524232 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.524090    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-metrics-client-ca\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.524232 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.524146    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-federate-client-tls\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.524232 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.524176    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-serving-certs-ca-bundle\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.524232 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.524204    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-client-tls\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.525027 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.524998    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-metrics-client-ca\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.525027 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.525017    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.525410 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.525386    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-serving-certs-ca-bundle\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.526885 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.526860    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.526975 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.526935    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.527060 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.527044    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-client-tls\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.527107 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.527045    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-federate-client-tls\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.536812 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.536762    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmlcb\" (UniqueName: \"kubernetes.io/projected/bd169cd6-f080-4fc4-82b4-cdb781a09640-kube-api-access-fmlcb\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.703693 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.703660    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:49:50.783346 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.783318    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5558d76459-nzh9f" event={"ID":"9b84a44a-cb54-47a0-834b-d5794c8d59ad","Type":"ContainerStarted","Data":"70d4859ab57d2dc451d265bca42cee959115e784869cfd5dd6b09473a5bdca57"}
Nov 04 03:49:50.998607 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:50.998582    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"]
Nov 04 03:49:51.001113 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:51.001087    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd169cd6_f080_4fc4_82b4_cdb781a09640.slice/crio-df1d4698f496fdee79503bbe70e41b6f4f79c2f5b1b5a76bdeed9479f3b6a759 WatchSource:0}: Error finding container df1d4698f496fdee79503bbe70e41b6f4f79c2f5b1b5a76bdeed9479f3b6a759: Status 404 returned error can't find the container with id df1d4698f496fdee79503bbe70e41b6f4f79c2f5b1b5a76bdeed9479f3b6a759
Nov 04 03:49:51.256693 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:51.256658    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:51.498792 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:51.498722    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:51.762013 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:51.761927    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:51.798456 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:51.798417    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerStarted","Data":"fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078"}
Nov 04 03:49:51.798650 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:51.798515    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerStarted","Data":"98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61"}
Nov 04 03:49:51.798650 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:51.798555    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerStarted","Data":"f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f"}
Nov 04 03:49:51.798650 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:51.798594    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerStarted","Data":"a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1"}
Nov 04 03:49:51.798650 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:51.798646    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerStarted","Data":"dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1"}
Nov 04 03:49:51.799763 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:51.799734    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b" event={"ID":"bd169cd6-f080-4fc4-82b4-cdb781a09640","Type":"ContainerStarted","Data":"df1d4698f496fdee79503bbe70e41b6f4f79c2f5b1b5a76bdeed9479f3b6a759"}
Nov 04 03:49:51.973490 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:51.973390    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:52.409766 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.409735    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-mv47f"]
Nov 04 03:49:52.413281 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.413256    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:49:52.421919 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.421896    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 03:49:52.422001 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.421954    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 03:49:52.422001 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.421979    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-pkj8n\""
Nov 04 03:49:52.423774 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.423752    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-mv47f"]
Nov 04 03:49:52.440664 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.440601    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83f09eac-bda3-430a-958a-2a9bef3ad8e0-tmp\") pod \"downloads-6dd789c8b7-mv47f\" (UID: \"83f09eac-bda3-430a-958a-2a9bef3ad8e0\") " pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:49:52.440772 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.440749    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgfvm\" (UniqueName: \"kubernetes.io/projected/83f09eac-bda3-430a-958a-2a9bef3ad8e0-kube-api-access-zgfvm\") pod \"downloads-6dd789c8b7-mv47f\" (UID: \"83f09eac-bda3-430a-958a-2a9bef3ad8e0\") " pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:49:52.541599 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.541560    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgfvm\" (UniqueName: \"kubernetes.io/projected/83f09eac-bda3-430a-958a-2a9bef3ad8e0-kube-api-access-zgfvm\") pod \"downloads-6dd789c8b7-mv47f\" (UID: \"83f09eac-bda3-430a-958a-2a9bef3ad8e0\") " pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:49:52.541756 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.541744    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83f09eac-bda3-430a-958a-2a9bef3ad8e0-tmp\") pod \"downloads-6dd789c8b7-mv47f\" (UID: \"83f09eac-bda3-430a-958a-2a9bef3ad8e0\") " pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:49:52.542198 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.542150    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83f09eac-bda3-430a-958a-2a9bef3ad8e0-tmp\") pod \"downloads-6dd789c8b7-mv47f\" (UID: \"83f09eac-bda3-430a-958a-2a9bef3ad8e0\") " pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:49:52.554606 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.554583    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgfvm\" (UniqueName: \"kubernetes.io/projected/83f09eac-bda3-430a-958a-2a9bef3ad8e0-kube-api-access-zgfvm\") pod \"downloads-6dd789c8b7-mv47f\" (UID: \"83f09eac-bda3-430a-958a-2a9bef3ad8e0\") " pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:49:52.727715 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.727643    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:49:52.810489 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.810457    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerStarted","Data":"ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2"}
Nov 04 03:49:52.848074 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:52.848023    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.571630248 podStartE2EDuration="6.848004107s" totalImagesPullingTime="4.264445487s" totalInitContainerRuntime="11.928372ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:52.845125924 +0000 UTC m=+63.004180812" watchObservedRunningTime="2026-11-04 03:49:52.848004107 +0000 UTC m=+63.007058971"
Nov 04 03:49:53.019263 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:53.019238    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-mv47f"]
Nov 04 03:49:53.022121 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:53.022084    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83f09eac_bda3_430a_958a_2a9bef3ad8e0.slice/crio-20b806b69a4b435d4b38f1fe2fec24efe2cd7e9cff9723239293b140b3aa3d18 WatchSource:0}: Error finding container 20b806b69a4b435d4b38f1fe2fec24efe2cd7e9cff9723239293b140b3aa3d18: Status 404 returned error can't find the container with id 20b806b69a4b435d4b38f1fe2fec24efe2cd7e9cff9723239293b140b3aa3d18
Nov 04 03:49:53.816009 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:53.815973    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-mv47f" event={"ID":"83f09eac-bda3-430a-958a-2a9bef3ad8e0","Type":"ContainerStarted","Data":"20b806b69a4b435d4b38f1fe2fec24efe2cd7e9cff9723239293b140b3aa3d18"}
Nov 04 03:49:53.817773 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:53.817734    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5558d76459-nzh9f" event={"ID":"9b84a44a-cb54-47a0-834b-d5794c8d59ad","Type":"ContainerStarted","Data":"5d8109ba51bb6b4c0f636a21e92a9040ea5bf9e581f3d3701a4d010a6e27f0b4"}
Nov 04 03:49:53.820122 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:53.820096    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b" event={"ID":"bd169cd6-f080-4fc4-82b4-cdb781a09640","Type":"ContainerStarted","Data":"8af685cf55e4503f5c429093733bf7bace4c167c435a1f82aa8737fa1215b55b"}
Nov 04 03:49:53.820217 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:53.820147    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b" event={"ID":"bd169cd6-f080-4fc4-82b4-cdb781a09640","Type":"ContainerStarted","Data":"3738becd34b831ac48aae4ce27e58b19dd964dc6a33923b404813e7f8f9b4c61"}
Nov 04 03:49:53.820217 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:53.820175    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b" event={"ID":"bd169cd6-f080-4fc4-82b4-cdb781a09640","Type":"ContainerStarted","Data":"83b3cd81611567b6f8442083e7622d2919c4b154d3e231b96ef1020630b3c851"}
Nov 04 03:49:53.840189 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:53.840141    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5558d76459-nzh9f" podStartSLOduration=2.187872832 podStartE2EDuration="4.840128199s" totalImagesPullingTime="2.652255367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:53.83964902 +0000 UTC m=+63.998703879" watchObservedRunningTime="2026-11-04 03:49:53.840128199 +0000 UTC m=+63.999183061"
Nov 04 03:49:53.867942 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:53.867905    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b" podStartSLOduration=1.620901647 podStartE2EDuration="3.86788912s" totalImagesPullingTime="2.246987473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:53.865515187 +0000 UTC m=+64.024570049" watchObservedRunningTime="2026-11-04 03:49:53.86788912 +0000 UTC m=+64.026943982"
Nov 04 03:49:54.333771 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:54.333737    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:54.562265 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:54.562221    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:56.075364 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.075315    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:56.075886 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.075434    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:56.075886 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.075469    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:56.078494 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.078471    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 03:49:56.078761 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.078743    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 03:49:56.079888 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.079864    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 03:49:56.089365 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.089340    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:56.089807 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.089790    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 03:49:56.091977 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.091956    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:56.097530 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.097507    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-p2j7s\""
Nov 04 03:49:56.098989 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.098972    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:56.102449 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.102430    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfbg\""
Nov 04 03:49:56.104569 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.104548    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:49:56.110237 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.110217    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:49:56.391936 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.391907    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-49g8t\""
Nov 04 03:49:56.398799 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.398556    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:49:56.405335 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.405310    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hn8bx"]
Nov 04 03:49:56.409592 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:56.409531    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e230e6_a451_475a_8b79_9d000021644b.slice/crio-45525316370e767ffde8e1e90ed74b9c697ca069f3c4f871c3ab12de7711e864 WatchSource:0}: Error finding container 45525316370e767ffde8e1e90ed74b9c697ca069f3c4f871c3ab12de7711e864: Status 404 returned error can't find the container with id 45525316370e767ffde8e1e90ed74b9c697ca069f3c4f871c3ab12de7711e864
Nov 04 03:49:56.416229 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.416206    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ksb9x"]
Nov 04 03:49:56.419550 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:56.419530    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode38ea962_08ee_44b8_8872_feec06d36ada.slice/crio-d78443c7ec157858a80c21fdc16549054b1438e5cb7f224a565b3e2ca983053b WatchSource:0}: Error finding container d78443c7ec157858a80c21fdc16549054b1438e5cb7f224a565b3e2ca983053b: Status 404 returned error can't find the container with id d78443c7ec157858a80c21fdc16549054b1438e5cb7f224a565b3e2ca983053b
Nov 04 03:49:56.686009 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.685980    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g97j2"]
Nov 04 03:49:56.687686 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:49:56.687651    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bbeb235_acfa_4f82_8edb_dc0727347f40.slice/crio-57be963177127e5a032bdbe0b05cdb98adfb4baaa8b85e750df8443783cad5ca WatchSource:0}: Error finding container 57be963177127e5a032bdbe0b05cdb98adfb4baaa8b85e750df8443783cad5ca: Status 404 returned error can't find the container with id 57be963177127e5a032bdbe0b05cdb98adfb4baaa8b85e750df8443783cad5ca
Nov 04 03:49:56.836696 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.836658    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g97j2" event={"ID":"4bbeb235-acfa-4f82-8edb-dc0727347f40","Type":"ContainerStarted","Data":"57be963177127e5a032bdbe0b05cdb98adfb4baaa8b85e750df8443783cad5ca"}
Nov 04 03:49:56.838455 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.838163    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ksb9x" event={"ID":"e38ea962-08ee-44b8-8872-feec06d36ada","Type":"ContainerStarted","Data":"d78443c7ec157858a80c21fdc16549054b1438e5cb7f224a565b3e2ca983053b"}
Nov 04 03:49:56.839510 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:56.839477    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hn8bx" event={"ID":"37e230e6-a451-475a-8b79-9d000021644b","Type":"ContainerStarted","Data":"45525316370e767ffde8e1e90ed74b9c697ca069f3c4f871c3ab12de7711e864"}
Nov 04 03:49:57.239712 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.238664    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c5d7fb6c9-6gqmq"]
Nov 04 03:49:57.242857 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.242835    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.257051 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.257023    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 04 03:49:57.259103 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.259079    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5d7fb6c9-6gqmq"]
Nov 04 03:49:57.259263 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.259253    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 04 03:49:57.259462 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.259351    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 04 03:49:57.259462 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.259391    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 04 03:49:57.259590 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.259464    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 04 03:49:57.259590 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.259567    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5s97h\""
Nov 04 03:49:57.289965 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.289930    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-oauth-config\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.290113 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.289976    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-service-ca\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.290183 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.290101    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5f4f\" (UniqueName: \"kubernetes.io/projected/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-kube-api-access-h5f4f\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.290239 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.290176    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-tmp\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.290239 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.290222    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-serving-cert\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.290341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.290263    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-config\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.290341 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.290293    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-oauth-serving-cert\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392218 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.391304    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-serving-cert\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392218 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.391350    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-config\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392218 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.391380    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-oauth-serving-cert\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392218 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.391402    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-oauth-config\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392218 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.391419    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-service-ca\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392218 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.391451    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5f4f\" (UniqueName: \"kubernetes.io/projected/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-kube-api-access-h5f4f\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392218 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.391495    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-tmp\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392218 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.391921    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-tmp\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392218 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.392204    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-config\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392790 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.392597    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-service-ca\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.392790 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.392669    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-oauth-serving-cert\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.395280 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.395245    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-serving-cert\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.399401 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.399378    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-oauth-config\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.411709 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.411659    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5f4f\" (UniqueName: \"kubernetes.io/projected/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-kube-api-access-h5f4f\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.570673 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.570573    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:49:57.882171 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.882089    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:57.894061 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:57.893966    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5d7fb6c9-6gqmq"]
Nov 04 03:49:58.089355 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:58.089318    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:58.636243 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:58.636090    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:58.876370 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:58.876056    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hn8bx" event={"ID":"37e230e6-a451-475a-8b79-9d000021644b","Type":"ContainerStarted","Data":"fccf2b52c952317681e2003adc651d4d49fe64c7a97c6da49148a15766df696d"}
Nov 04 03:49:58.876370 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:58.876125    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hn8bx" event={"ID":"37e230e6-a451-475a-8b79-9d000021644b","Type":"ContainerStarted","Data":"d34ccfaac538252c5f80416da0c2c2580337bbbad4534ddb5ffda9684b9c5124"}
Nov 04 03:49:58.879270 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:58.879240    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5d7fb6c9-6gqmq" event={"ID":"1c907e96-5900-4ede-b7a8-dbf6e6a092d0","Type":"ContainerStarted","Data":"c44fc83f2a2cdda3fbedea17326a62e08b99b077a82879132f67aa13bc444078"}
Nov 04 03:49:58.905177 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:58.903455    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hn8bx" podStartSLOduration=66.142838082 podStartE2EDuration="1m7.903437495s" totalImagesPullingTime="1.760599413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:49:58.900652094 +0000 UTC m=+69.059706976" watchObservedRunningTime="2026-11-04 03:49:58.903437495 +0000 UTC m=+69.062492380"
Nov 04 03:49:58.906654 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:58.906609    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:49:59.181458 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:49:59.181391    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:00.208112 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.208077    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-579797c45f-dt5hr"]
Nov 04 03:50:00.212851 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.212812    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.225151 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.225118    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-579797c45f-dt5hr"]
Nov 04 03:50:00.230267 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.230228    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 04 03:50:00.340349 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.340319    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:00.365114 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.365075    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-oauth-serving-cert\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.365227 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.365120    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58j2j\" (UniqueName: \"kubernetes.io/projected/99a5bcad-5137-4a71-8130-18ab8f851b71-kube-api-access-58j2j\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.365227 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.365151    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-service-ca\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.365227 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.365183    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-trusted-ca-bundle\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.365331 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.365227    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-console-config\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.365331 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.365298    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-serving-cert\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.365331 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.365321    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a5bcad-5137-4a71-8130-18ab8f851b71-tmp\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.365420 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.365359    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-oauth-config\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.466711 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.466638    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-oauth-serving-cert\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.466711 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.466676    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58j2j\" (UniqueName: \"kubernetes.io/projected/99a5bcad-5137-4a71-8130-18ab8f851b71-kube-api-access-58j2j\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.466711 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.466699    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-service-ca\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.466984 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.466716    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-trusted-ca-bundle\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.466984 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.466749    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-console-config\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.466984 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.466815    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-serving-cert\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.466984 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.466837    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a5bcad-5137-4a71-8130-18ab8f851b71-tmp\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.466984 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.466879    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-oauth-config\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.468493 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.468146    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-trusted-ca-bundle\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.468493 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.468181    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-oauth-serving-cert\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.468493 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.468309    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-console-config\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.468737 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.468577    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a5bcad-5137-4a71-8130-18ab8f851b71-tmp\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.468737 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.468683    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-service-ca\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.469920 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.469897    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-oauth-config\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.471196 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.471168    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-serving-cert\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.481943 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.481917    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58j2j\" (UniqueName: \"kubernetes.io/projected/99a5bcad-5137-4a71-8130-18ab8f851b71-kube-api-access-58j2j\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.529125 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.529105    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:00.551013 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.550899    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:00.831042 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.831010    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-579797c45f-dt5hr"]
Nov 04 03:50:00.833768 ip-10-0-129-253 kubenswrapper[2573]: W1104 03:50:00.833735    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a5bcad_5137_4a71_8130_18ab8f851b71.slice/crio-bffd53b9b2b4de941a8bd7a30e50852b1b76b07f7e444242dd71fa4ae587ee90 WatchSource:0}: Error finding container bffd53b9b2b4de941a8bd7a30e50852b1b76b07f7e444242dd71fa4ae587ee90: Status 404 returned error can't find the container with id bffd53b9b2b4de941a8bd7a30e50852b1b76b07f7e444242dd71fa4ae587ee90
Nov 04 03:50:00.892304 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:00.892215    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579797c45f-dt5hr" event={"ID":"99a5bcad-5137-4a71-8130-18ab8f851b71","Type":"ContainerStarted","Data":"bffd53b9b2b4de941a8bd7a30e50852b1b76b07f7e444242dd71fa4ae587ee90"}
Nov 04 03:50:02.903612 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:02.903544    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g97j2" event={"ID":"4bbeb235-acfa-4f82-8edb-dc0727347f40","Type":"ContainerStarted","Data":"d4702572b074c936682caee8646b5a1c05be6c4dbee4c2fbb3a44307b5e8d515"}
Nov 04 03:50:02.904103 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:02.903676    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:50:02.906114 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:02.906077    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5d7fb6c9-6gqmq" event={"ID":"1c907e96-5900-4ede-b7a8-dbf6e6a092d0","Type":"ContainerStarted","Data":"57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192"}
Nov 04 03:50:02.923491 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:02.923432    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g97j2" podStartSLOduration=67.355604467 podStartE2EDuration="1m12.923417031s" totalImagesPullingTime="5.567812564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:50:02.922501566 +0000 UTC m=+73.081556427" watchObservedRunningTime="2026-11-04 03:50:02.923417031 +0000 UTC m=+73.082471891"
Nov 04 03:50:02.947852 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:02.947804    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5c5d7fb6c9-6gqmq" podStartSLOduration=1.454808863 podStartE2EDuration="5.947791212s" totalImagesPullingTime="4.492982349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:50:02.946979335 +0000 UTC m=+73.106034195" watchObservedRunningTime="2026-11-04 03:50:02.947791212 +0000 UTC m=+73.106846070"
Nov 04 03:50:03.284332 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:03.284246    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:03.495020 ip-10-0-129-253 kubenswrapper[2573]: I1104 03:50:03.494979    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:05.604986 ip-10-0-129-253 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 03:50:05.616107 ip-10-0-129-253 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 03:50:05.616326 ip-10-0-129-253 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 03:50:05.616386 ip-10-0-129-253 systemd[1]: kubelet.service: Consumed 4.119s CPU time.
Nov 04 03:50:05.635171 ip-10-0-129-253 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 03:50:05.696247 ip-10-0-129-253 kubenswrapper[6751]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:50:05.696247 ip-10-0-129-253 kubenswrapper[6751]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 03:50:05.696247 ip-10-0-129-253 kubenswrapper[6751]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:50:05.696247 ip-10-0-129-253 kubenswrapper[6751]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 03:50:05.698369 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698290    6751 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:50:05.698369 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698367    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:50:05.698369 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698372    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:50:05.698369 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698376    6751 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698379    6751 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698382    6751 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698385    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698388    6751 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698391    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698393    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698396    6751 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698399    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698402    6751 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698405    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698407    6751 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698410    6751 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698413    6751 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698416    6751 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698418    6751 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698421    6751 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698423    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698426    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:50:05.698522 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698429    6751 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698432    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698434    6751 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698437    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698447    6751 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698450    6751 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698453    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698456    6751 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698458    6751 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698460    6751 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698463    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698466    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698469    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698471    6751 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698474    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698478    6751 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698482    6751 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698485    6751 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698488    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698491    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:50:05.699023 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698494    6751 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698497    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698499    6751 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698502    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698505    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698507    6751 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698510    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698513    6751 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698515    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698518    6751 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698521    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698525    6751 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698528    6751 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698531    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698534    6751 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698536    6751 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698539    6751 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698542    6751 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698546    6751 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:50:05.699521 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698548    6751 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698551    6751 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698555    6751 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698560    6751 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698563    6751 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698566    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698568    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698571    6751 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698574    6751 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698576    6751 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698579    6751 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698581    6751 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698584    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698587    6751 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698589    6751 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698592    6751 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698594    6751 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698597    6751 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698600    6751 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698603    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:50:05.700016 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698605    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698608    6751 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698611    6751 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698614    6751 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698631    6751 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698636    6751 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698640    6751 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698643    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698646    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698648    6751 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698651    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698662    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698665    6751 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698667    6751 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698670    6751 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698673    6751 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698675    6751 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698678    6751 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698681    6751 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698683    6751 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:50:05.700512 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698686    6751 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698688    6751 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698691    6751 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698694    6751 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.698697    6751 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699466    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699472    6751 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699476    6751 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699480    6751 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699484    6751 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699488    6751 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699491    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699494    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699496    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699499    6751 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699501    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699504    6751 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699506    6751 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699509    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699511    6751 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:50:05.701073 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699514    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699517    6751 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699521    6751 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699524    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699528    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699531    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699535    6751 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699538    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699540    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699543    6751 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699545    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699548    6751 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699550    6751 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699553    6751 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699556    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699558    6751 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699561    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699564    6751 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699567    6751 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:50:05.701599 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699570    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699573    6751 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699575    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699578    6751 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699580    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699583    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699585    6751 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699588    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699591    6751 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699593    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699596    6751 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699598    6751 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699601    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699603    6751 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699606    6751 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699608    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699611    6751 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699613    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699632    6751 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:50:05.702106 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699636    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699639    6751 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699642    6751 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699645    6751 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699647    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699650    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699653    6751 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699655    6751 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699658    6751 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699661    6751 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699664    6751 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699668    6751 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699672    6751 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699676    6751 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699680    6751 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699683    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699688    6751 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699692    6751 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699696    6751 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:50:05.702576 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699699    6751 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699703    6751 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699707    6751 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699710    6751 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699714    6751 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699718    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699723    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699727    6751 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699731    6751 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699735    6751 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699739    6751 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699743    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699747    6751 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699752    6751 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699757    6751 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699760    6751 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699765    6751 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699769    6751 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699774    6751 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699778    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:50:05.703071 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699783    6751 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699787    6751 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699792    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699795    6751 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699798    6751 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699800    6751 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699803    6751 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699806    6751 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699809    6751 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699811    6751 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699814    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699817    6751 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699820    6751 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.699822    6751 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700307    6751 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700312    6751 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700315    6751 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700318    6751 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700324    6751 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700327    6751 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:50:05.703545 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700329    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700332    6751 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700335    6751 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700337    6751 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700340    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700343    6751 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700346    6751 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700349    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700352    6751 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700354    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700357    6751 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700361    6751 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700366    6751 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700370    6751 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700374    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700377    6751 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700380    6751 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700383    6751 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700386    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:50:05.704083 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700389    6751 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700391    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700394    6751 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700397    6751 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700399    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700402    6751 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700404    6751 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700407    6751 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700410    6751 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700412    6751 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700415    6751 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700421    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700423    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700426    6751 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700429    6751 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700431    6751 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700434    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700437    6751 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700439    6751 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700443    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:50:05.704780 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700445    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700448    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700450    6751 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700453    6751 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700458    6751 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700462    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700465    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700468    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700470    6751 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700473    6751 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700476    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700478    6751 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700481    6751 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700483    6751 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700486    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700488    6751 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700491    6751 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700493    6751 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700496    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:50:05.705587 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700498    6751 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700501    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700503    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700506    6751 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700510    6751 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700512    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700515    6751 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700518    6751 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700521    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700523    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700526    6751 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700529    6751 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700531    6751 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700535    6751 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700537    6751 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700540    6751 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700542    6751 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700545    6751 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700547    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700550    6751 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:50:05.706179 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700552    6751 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700555    6751 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700557    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700560    6751 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700562    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700565    6751 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700567    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700570    6751 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700572    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700575    6751 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700577    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700580    6751 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700582    6751 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700585    6751 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700587    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700590    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700594    6751 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700597    6751 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700600    6751 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:50:05.706693 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700603    6751 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700606    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.700608    6751 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700720    6751 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700728    6751 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700734    6751 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700739    6751 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700743    6751 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700747    6751 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700751    6751 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700755    6751 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700758    6751 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700761    6751 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700765    6751 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700768    6751 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700772    6751 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700775    6751 flags.go:64] FLAG: --cgroup-root=""
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700778    6751 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700781    6751 flags.go:64] FLAG: --client-ca-file=""
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700784    6751 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700786    6751 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700790    6751 flags.go:64] FLAG: --cluster-domain=""
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700794    6751 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700797    6751 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 03:50:05.707217 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700800    6751 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700807    6751 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700811    6751 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700814    6751 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700817    6751 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700821    6751 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700825    6751 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700828    6751 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700831    6751 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700834    6751 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700837    6751 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700841    6751 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700845    6751 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700848    6751 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700851    6751 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700854    6751 flags.go:64] FLAG: --enable-server="true"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700858    6751 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700862    6751 flags.go:64] FLAG: --event-burst="100"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700865    6751 flags.go:64] FLAG: --event-qps="50"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700868    6751 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700871    6751 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700874    6751 flags.go:64] FLAG: --eviction-hard=""
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700878    6751 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700881    6751 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700884    6751 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 03:50:05.707804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700887    6751 flags.go:64] FLAG: --eviction-soft=""
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700890    6751 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700893    6751 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700896    6751 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700899    6751 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700902    6751 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700906    6751 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700909    6751 flags.go:64] FLAG: --feature-gates=""
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700919    6751 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700922    6751 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700925    6751 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700929    6751 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700933    6751 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700936    6751 flags.go:64] FLAG: --help="false"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700939    6751 flags.go:64] FLAG: --hostname-override="ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700942    6751 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700945    6751 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700948    6751 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700952    6751 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700955    6751 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700958    6751 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700961    6751 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700964    6751 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 03:50:05.708417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700967    6751 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700970    6751 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700974    6751 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700977    6751 flags.go:64] FLAG: --kube-reserved=""
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700980    6751 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700983    6751 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700986    6751 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700989    6751 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700991    6751 flags.go:64] FLAG: --lock-file=""
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700994    6751 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.700998    6751 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701001    6751 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701006    6751 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701009    6751 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701012    6751 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701015    6751 flags.go:64] FLAG: --logging-format="text"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701018    6751 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701023    6751 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701026    6751 flags.go:64] FLAG: --manifest-url=""
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701029    6751 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701033    6751 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701038    6751 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701042    6751 flags.go:64] FLAG: --max-pods="110"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701045    6751 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701048    6751 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 03:50:05.708994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701051    6751 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701055    6751 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701058    6751 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701061    6751 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701064    6751 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701070    6751 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701073    6751 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701076    6751 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701080    6751 flags.go:64] FLAG: --pod-cidr=""
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701083    6751 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701086    6751 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701090    6751 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701093    6751 flags.go:64] FLAG: --port="10250"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701096    6751 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701099    6751 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-02ee2cea9e1beaecf"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701102    6751 flags.go:64] FLAG: --qos-reserved=""
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701105    6751 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701108    6751 flags.go:64] FLAG: --register-node="true"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701111    6751 flags.go:64] FLAG: --register-with-taints=""
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701115    6751 flags.go:64] FLAG: --registry-burst="10"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701118    6751 flags.go:64] FLAG: --registry-qps="5"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701121    6751 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701124    6751 flags.go:64] FLAG: --reserved-memory=""
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701128    6751 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701133    6751 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701136    6751 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 03:50:05.709605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701139    6751 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701142    6751 flags.go:64] FLAG: --runonce="false"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701146    6751 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701149    6751 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701152    6751 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701155    6751 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701158    6751 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701162    6751 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701165    6751 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701168    6751 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701171    6751 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701174    6751 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701178    6751 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701181    6751 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701184    6751 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701187    6751 flags.go:64] FLAG: --system-cgroups=""
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701190    6751 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701196    6751 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701198    6751 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701201    6751 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701205    6751 flags.go:64] FLAG: --tls-min-version=""
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701208    6751 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701211    6751 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701214    6751 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701217    6751 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 03:50:05.710274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701220    6751 flags.go:64] FLAG: --v="2"
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701225    6751 flags.go:64] FLAG: --version="false"
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701229    6751 flags.go:64] FLAG: --vmodule=""
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701233    6751 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701236    6751 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701362    6751 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701366    6751 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701369    6751 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701372    6751 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701376    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701378    6751 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701381    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701384    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701386    6751 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701389    6751 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701392    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701395    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701397    6751 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701400    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701403    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701405    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:50:05.710984 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701408    6751 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701411    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701413    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701416    6751 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701419    6751 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701422    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701424    6751 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701428    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701430    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701432    6751 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701435    6751 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701438    6751 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701440    6751 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701443    6751 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701445    6751 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701448    6751 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701452    6751 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701454    6751 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701457    6751 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701460    6751 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:50:05.711495 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701465    6751 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701467    6751 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701470    6751 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701473    6751 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701475    6751 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701478    6751 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701480    6751 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701483    6751 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701486    6751 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701488    6751 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701491    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701493    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701496    6751 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701498    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701501    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701503    6751 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701507    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701509    6751 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701512    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:50:05.712050 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701514    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701517    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701520    6751 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701522    6751 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701524    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701527    6751 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701530    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701533    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701537    6751 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701542    6751 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701546    6751 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701551    6751 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701555    6751 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701562    6751 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701565    6751 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701568    6751 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701572    6751 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701576    6751 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701579    6751 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:50:05.712584 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701583    6751 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701586    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701588    6751 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701591    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701594    6751 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701596    6751 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701599    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701602    6751 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701606    6751 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701610    6751 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701613    6751 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701632    6751 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701636    6751 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701639    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701641    6751 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701644    6751 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701646    6751 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701649    6751 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701652    6751 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701654    6751 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:50:05.713086 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701657    6751 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701660    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701664    6751 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701667    6751 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701669    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701672    6751 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701675    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701678    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701680    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701683    6751 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701686    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701688    6751 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.701717    6751 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701753    6751 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701757    6751 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701760    6751 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:50:05.713581 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701763    6751 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701766    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701768    6751 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701771    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701773    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701776    6751 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701779    6751 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701781    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701784    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701787    6751 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701790    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701792    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701795    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701797    6751 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701800    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701802    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701805    6751 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701808    6751 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701810    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:50:05.714092 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701814    6751 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701817    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701820    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701823    6751 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701826    6751 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701828    6751 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701831    6751 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701834    6751 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701836    6751 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701839    6751 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701841    6751 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701844    6751 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701846    6751 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701849    6751 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701852    6751 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701855    6751 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701857    6751 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701860    6751 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701862    6751 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701865    6751 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:50:05.714729 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701867    6751 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701870    6751 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701873    6751 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701876    6751 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701878    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701881    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701883    6751 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701886    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701888    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701891    6751 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701894    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701896    6751 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701901    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701904    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701906    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701909    6751 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701912    6751 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701914    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701917    6751 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:50:05.715367 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701919    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701922    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701925    6751 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701927    6751 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701930    6751 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701933    6751 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701935    6751 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701938    6751 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701941    6751 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701944    6751 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701947    6751 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701951    6751 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701954    6751 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701957    6751 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701959    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701962    6751 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701965    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701968    6751 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701970    6751 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:50:05.715900 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701973    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701976    6751 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701979    6751 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701983    6751 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701986    6751 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701989    6751 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701994    6751 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701996    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.701999    6751 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702001    6751 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702004    6751 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702007    6751 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702009    6751 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702012    6751 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702015    6751 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702017    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702020    6751 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702022    6751 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702025    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702027    6751 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:50:05.716353 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702030    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702032    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702035    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702037    6751 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702040    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.702043    6751 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.707381    6751 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.707395    6751 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.707419    6751 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.707423    6751 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707483    6751 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707489    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707492    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707495    6751 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707498    6751 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707501    6751 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707504    6751 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707507    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707511    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:50:05.716862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707513    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707516    6751 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707519    6751 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707522    6751 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707524    6751 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707527    6751 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707530    6751 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707533    6751 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707535    6751 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707538    6751 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707541    6751 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707543    6751 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707546    6751 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707548    6751 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707551    6751 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707553    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707556    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707559    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707562    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707564    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:50:05.717324 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707567    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707569    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707572    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707575    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707578    6751 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707581    6751 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707583    6751 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707586    6751 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707588    6751 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707591    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707594    6751 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707596    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707599    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707602    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707605    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707607    6751 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707610    6751 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707633    6751 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707637    6751 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707639    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:50:05.717849 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707642    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707645    6751 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707648    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707652    6751 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707656    6751 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707659    6751 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707663    6751 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707665    6751 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707668    6751 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707670    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707673    6751 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707675    6751 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707678    6751 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707681    6751 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707683    6751 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707686    6751 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707689    6751 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707691    6751 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707694    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707696    6751 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:50:05.718382 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707699    6751 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707702    6751 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707704    6751 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707707    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707710    6751 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707713    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707715    6751 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707718    6751 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707720    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707723    6751 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707731    6751 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707734    6751 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707737    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707739    6751 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707742    6751 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707744    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707747    6751 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707750    6751 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707752    6751 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:50:05.719066 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707754    6751 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707757    6751 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707760    6751 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707762    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707764    6751 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707767    6751 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707770    6751 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707773    6751 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707777    6751 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707780    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707783    6751 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707785    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707788    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707791    6751 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707793    6751 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707796    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707798    6751 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:50:05.719862 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707801    6751 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.707830    6751 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707976    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707982    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707985    6751 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707988    6751 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707990    6751 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707994    6751 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707997    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.707999    6751 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708002    6751 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708005    6751 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708008    6751 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708011    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708013    6751 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708016    6751 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:50:05.720398 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708018    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708021    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708023    6751 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708027    6751 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708030    6751 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708035    6751 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708038    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708040    6751 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708043    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708046    6751 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708048    6751 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708051    6751 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708054    6751 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708056    6751 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708059    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708061    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708064    6751 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708066    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708069    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708071    6751 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:50:05.720812 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708074    6751 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708077    6751 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708079    6751 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708082    6751 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708086    6751 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708088    6751 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708091    6751 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708093    6751 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708096    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708099    6751 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708102    6751 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708104    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708108    6751 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708113    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708115    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708118    6751 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708121    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708125    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708127    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:50:05.721370 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708130    6751 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708135    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708138    6751 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708140    6751 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708143    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708146    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708149    6751 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708151    6751 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708154    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708156    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708159    6751 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708161    6751 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708164    6751 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708167    6751 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708170    6751 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708172    6751 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708175    6751 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708178    6751 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708181    6751 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:50:05.721950 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708184    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708186    6751 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708189    6751 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708192    6751 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708194    6751 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708197    6751 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708199    6751 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708202    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708205    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708207    6751 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708210    6751 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708214    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708216    6751 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708219    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708222    6751 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708225    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708228    6751 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708230    6751 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708233    6751 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708235    6751 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:50:05.722442 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708238    6751 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708241    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708243    6751 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708246    6751 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708248    6751 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708251    6751 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708254    6751 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708256    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708259    6751 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708261    6751 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708264    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708266    6751 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708270    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708274    6751 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.708297    6751 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 03:50:05.722957 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708331    6751 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708335    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708338    6751 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708341    6751 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708344    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708347    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708350    6751 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708352    6751 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708357    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708360    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708362    6751 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708365    6751 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708369    6751 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708371    6751 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708374    6751 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708377    6751 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708380    6751 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708383    6751 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708386    6751 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 03:50:05.723366 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708388    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708391    6751 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708394    6751 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708397    6751 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708399    6751 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708402    6751 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708404    6751 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708407    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708410    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708413    6751 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708415    6751 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708418    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708421    6751 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708424    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708427    6751 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708430    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708433    6751 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708435    6751 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708438    6751 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708440    6751 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 03:50:05.723829 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708443    6751 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708446    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708449    6751 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708451    6751 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708454    6751 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708457    6751 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708460    6751 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708463    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708465    6751 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708468    6751 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708470    6751 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708473    6751 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708475    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708480    6751 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708484    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708486    6751 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708490    6751 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708493    6751 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708496    6751 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 03:50:05.724331 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708499    6751 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708501    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708504    6751 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708506    6751 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708509    6751 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708512    6751 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708515    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708517    6751 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708520    6751 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708522    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708525    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708527    6751 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708530    6751 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708532    6751 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708535    6751 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708538    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708541    6751 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708543    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708546    6751 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708548    6751 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 03:50:05.724848 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708551    6751 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708553    6751 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708556    6751 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708559    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708561    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708564    6751 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708566    6751 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708569    6751 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708572    6751 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708574    6751 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708577    6751 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708579    6751 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708582    6751 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708584    6751 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708587    6751 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708590    6751 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708592    6751 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708595    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708598    6751 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 03:50:05.725337 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708601    6751 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708603    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708608    6751 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708612    6751 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708615    6751 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708631    6751 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708636    6751 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708640    6751 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:05.708643    6751 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.708692    6751 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.709599    6751 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.709663    6751 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.709838    6751 server.go:1014] "Starting client certificate rotation"
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.709975    6751 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.710022    6751 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 03:43:50 +0000 UTC" deadline="2028-07-04 13:05:42.112171138 +0000 UTC"
Nov 04 03:50:05.725802 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.710056    6751 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14601h15m36.402116668s"
Nov 04 03:50:05.726184 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.710278    6751 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:50:05.726184 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.711338    6751 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 03:50:05.726184 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.712520    6751 log.go:25] "Validated CRI v1 runtime API"
Nov 04 03:50:05.726184 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.714338    6751 log.go:25] "Validated CRI v1 image API"
Nov 04 03:50:05.726184 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.715163    6751 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 03:50:05.726184 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.717307    6751 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aaa51dfe-3927-45e7-b95d-c052a1b66fda:/dev/nvme0n1p3 e8155bf8-bfe0-4ce0-97ca-af858267fde6:/dev/nvme0n1p4]
Nov 04 03:50:05.726377 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.717325    6751 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/12f7f6d603150db4d1f8d7d6504361a27aa4ec238a45f6fede6d616e652ee6ee/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/12f7f6d603150db4d1f8d7d6504361a27aa4ec238a45f6fede6d616e652ee6ee/userdata/shm major:0 minor:265 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/20b806b69a4b435d4b38f1fe2fec24efe2cd7e9cff9723239293b140b3aa3d18/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/20b806b69a4b435d4b38f1fe2fec24efe2cd7e9cff9723239293b140b3aa3d18/userdata/shm major:0 minor:376 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2fc3d49c43c6a5cdaf40e75b563deb9a951dd7032324a2304bc5dcb4f3988fc8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2fc3d49c43c6a5cdaf40e75b563deb9a951dd7032324a2304bc5dcb4f3988fc8/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/31e5a426a6112b4597dd55e9d617496d1de8aeb96583b41aaccc5f81167af172/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/31e5a426a6112b4597dd55e9d617496d1de8aeb96583b41aaccc5f81167af172/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/36508a0c4387cbd148e95192f28fdfd39b34a9db94d58ac0e986eac0b68285cc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/36508a0c4387cbd148e95192f28fdfd39b34a9db94d58ac0e986eac0b68285cc/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3dbc6454498ef5643c21f406401cd53cea0aa791e9aad6e6d5a996202ed5d896/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3dbc6454498ef5643c21f406401cd53cea0aa791e9aad6e6d5a996202ed5d896/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3f25a6490432e4fec55af20c26b754a466bacdfec844667aec4125087850f471/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3f25a6490432e4fec55af20c26b754a466bacdfec844667aec4125087850f471/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/45525316370e767ffde8e1e90ed74b9c697ca069f3c4f871c3ab12de7711e864/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/45525316370e767ffde8e1e90ed74b9c697ca069f3c4f871c3ab12de7711e864/userdata/shm major:0 minor:406 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/53db76be9492a37e78f045cf9979b5cb6912ead66e9b291d45d5a8fb63c4b6ea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/53db76be9492a37e78f045cf9979b5cb6912ead66e9b291d45d5a8fb63c4b6ea/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/57be963177127e5a032bdbe0b05cdb98adfb4baaa8b85e750df8443783cad5ca/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/57be963177127e5a032bdbe0b05cdb98adfb4baaa8b85e750df8443783cad5ca/userdata/shm major:0 minor:411 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/594c966f892813491d1f7cc9e017421c2dc1870030368c603635501ced8aa703/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/594c966f892813491d1f7cc9e017421c2dc1870030368c603635501ced8aa703/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/67f0de77c5a45d70de89ea8b34a7ba039802bafdab6a37f94a42a3b4cbc81291/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/67f0de77c5a45d70de89ea8b34a7ba039802bafdab6a37f94a42a3b4cbc81291/userdata/shm major:0 minor:259 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/70d4859ab57d2dc451d265bca42cee959115e784869cfd5dd6b09473a5bdca57/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/70d4859ab57d2dc451d265bca42cee959115e784869cfd5dd6b09473a5bdca57/userdata/shm major:0 minor:331 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/786aaf26a4bf4d0051e0e886149fd2053217ddfa20d268c7d55c33fd43434cd8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/786aaf26a4bf4d0051e0e886149fd2053217ddfa20d268c7d55c33fd43434cd8/userdata/shm major:0 minor:261 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7eb7017d429d5a601074116d7db76253931352236a7d83f7277bcc72e1a25745/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7eb7017d429d5a601074116d7db76253931352236a7d83f7277bcc72e1a25745/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9720f4fd3d9c516912ed18b22275262cd8dfc63cda1d7b06c7e23eb67e603b4f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9720f4fd3d9c516912ed18b22275262cd8dfc63cda1d7b06c7e23eb67e603b4f/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9df2c3649bb0878ed89a04f42679d1ef8f81eb62a749d84e255daccabfeede60/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9df2c3649bb0878ed89a04f42679d1ef8f81eb62a749d84e255daccabfeede60/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a4d63b7b1d745fea45bfcef6030a7b315629c44a9f1861a86b5fa8a5e9870456/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a4d63b7b1d745fea45bfcef6030a7b315629c44a9f1861a86b5fa8a5e9870456/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bffd53b9b2b4de941a8bd7a30e50852b1b76b07f7e444242dd71fa4ae587ee90/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bffd53b9b2b4de941a8bd7a30e50852b1b76b07f7e444242dd71fa4ae587ee90/userdata/shm major:0 minor:438 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c44fc83f2a2cdda3fbedea17326a62e08b99b077a82879132f67aa13bc444078/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c44fc83f2a2cdda3fbedea17326a62e08b99b077a82879132f67aa13bc444078/userdata/shm major:0 minor:420 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d0d74ca4f35221090e986f4163b534d866537d1f570aefd78de83725ee788a09/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d0d74ca4f35221090e986f4163b534d866537d1f570aefd78de83725ee788a09/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d78443c7ec157858a80c21fdc16549054b1438e5cb7f224a565b3e2ca983053b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d78443c7ec157858a80c21fdc16549054b1438e5cb7f224a565b3e2ca983053b/userdata/shm major:0 minor:405 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/df1d4698f496fdee79503bbe70e41b6f4f79c2f5b1b5a76bdeed9479f3b6a759/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/df1d4698f496fdee79503bbe70e41b6f4f79c2f5b1b5a76bdeed9479f3b6a759/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e433bbe80c3f57d3441236b9113fdbe7b990e68833e7f23814978187501c1f45/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e433bbe80c3f57d3441236b9113fdbe7b990e68833e7f23814978187501c1f45/userdata/shm major:0 minor:292 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f276f8204b9a52b02e59f475a275f7e61cc344927976a6479b95da64e8e16292/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f276f8204b9a52b02e59f475a275f7e61cc344927976a6479b95da64e8e16292/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0982ff5a-da06-4180-9838-6bfc5f2deba0/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/0982ff5a-da06-4180-9838-6bfc5f2deba0/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0982ff5a-da06-4180-9838-6bfc5f2deba0/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/0982ff5a-da06-4180-9838-6bfc5f2deba0/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0982ff5a-da06-4180-9838-6bfc5f2deba0/volumes/kubernetes.io~projected/kube-api-access-vljp6:{mountpoint:/var/lib/kubelet/pods/0982ff5a-da06-4180-9838-6bfc5f2deba0/volumes/kubernetes.io~projected/kube-api-access-vljp6 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bb68137-3e53-42d0-a9fd-e6cddab3346c/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/0bb68137-3e53-42d0-a9fd-e6cddab3346c/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bb68137-3e53-42d0-a9fd-e6cddab3346c/volumes/kubernetes.io~projected/kube-api-access-jdbcn:{mountpoint:/var/lib/kubelet/pods/0bb68137-3e53-42d0-a9fd-e6cddab3346c/volumes/kubernetes.io~projected/kube-api-access-jdbcn major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bb68137-3e53-42d0-a9fd-e6cddab3346c/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/0bb68137-3e53-42d0-a9fd-e6cddab3346c/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c907e96-5900-4ede-b7a8-dbf6e6a092d0/volumes/kubernetes.io~projected/kube-api-access-h5f4f:{mountpoint:/var/lib/kubelet/pods/1c907e96-5900-4ede-b7a8-dbf6e6a092d0/volumes/kubernetes.io~projected/kube-api-access-h5f4f major:0 minor:419 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c907e96-5900-4ede-b7a8-dbf6e6a092d0/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/1c907e96-5900-4ede-b7a8-dbf6e6a092d0/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:417 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c907e96-5900-4ede-b7a8-dbf6e6a092d0/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/1c907e96-5900-4ede-b7a8-dbf6e6a092d0/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:418 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/volumes/kubernetes.io~projected/kube-api-access-hv5wd:{mountpoint:/var/lib/kubelet/pods/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/volumes/kubernetes.io~projected/kube-api-access-hv5wd major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:255 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:252 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/37e230e6-a451-475a-8b79-9d000021644b/volumes/kubernetes.io~projected/kube-api-access-bsqvw:{mountpoint:/var/lib/kubelet/pods/37e230e6-a451-475a-8b79-9d000021644b/volumes/kubernetes.io~projected/kube-api-access-bsqvw major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/37e230e6-a451-475a-8b79-9d000021644b/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/37e230e6-a451-475a-8b79-9d000021644b/volumes/kubernetes.io~secret/metrics-certs major:0 minor:403 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4bbeb235-acfa-4f82-8edb-dc0727347f40/volumes/kubernetes.io~projected/kube-api-access-ggjv6:{mountpoint:/var/lib/kubelet/pods/4bbeb235-acfa-4f82-8edb-dc0727347f40/volumes/kubernetes.io~projected/kube-api-access-ggjv6 major:0 minor:404 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e/volumes/kubernetes.io~projected/kube-api-access-cgx6l:{mountpoint:/var/lib/kubelet/pods/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e/volumes/kubernetes.io~projected/kube-api-access-cgx6l major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4f25400e-aea9-43e3-b17f-0a68a76a4632/volumes/kubernetes.io~projected/kube-api-access-qsvkq:{mountpoint:/var/lib/kubelet/pods/4f25400e-aea9-43e3-b17f-0a68a76a4632/volumes/kubernetes.io~projected/kube-api-access-qsvkq major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4f25400e-aea9-43e3-b17f-0a68a76a4632/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/4f25400e-aea9-43e3-b17f-0a68a76a4632/volumes/kubernetes.io~secret/metrics-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68df7a49-2cdd-46c3-8730-9447708be784/volumes/kubernetes.io~projected/kube-api-access-nt9sb:{mountpoint:/var/lib/kubelet/pods/68df7a49-2cdd-46c3-8730-9447708be784/volumes/kubernetes.io~projected/kube-api-access-nt9sb major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68df7a49-2cdd-46c3-8730-9447708be784/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/68df7a49-2cdd-46c3-8730-9447708be784/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74f52bd8-5314-4ba0-8834-81e86a91c1f6/volumes/kubernetes.io~projected/kube-api-access-sqkww:{mountpoint:/var/lib/kubelet/pods/74f52bd8-5314-4ba0-8834-81e86a91c1f6/volumes/kubernetes.io~projected/kube-api-access-sqkww major:0 minor:256 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74f52bd8-5314-4ba0-8834-81e86a91c1f6/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/74f52bd8-5314-4ba0-8834-81e86a91c1f6/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:253 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74f52bd8-5314-4ba0-8834-81e86a91c1f6/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/74f52bd8-5314-4ba0-8834-81e86a91c1f6/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:251 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83f09eac-bda3-430a-958a-2a9bef3ad8e0/volumes/kubernetes.io~projected/kube-api-access-zgfvm:{mountpoint:/var/lib/kubelet/pods/83f09eac-bda3-430a-958a-2a9bef3ad8e0/volumes/kubernetes.io~projected/kube-api-access-zgfvm major:0 minor:372 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/86f60625-8953-4f46-aa16-cc1d9142c4df/volumes/kubernetes.io~projected/kube-api-access-2qpr7:{mountpoint:/var/lib/kubelet/pods/86f60625-8953-4f46-aa16-cc1d9142c4df/volumes/kubernetes.io~projected/kube-api-access-2qpr7 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9062ed54-5f71-4f96-8274-20ab5a38a9a0/volumes/kubernetes.io~projected/kube-api-access-7jzxp:{mountpoint:/var/lib/kubelet/pods/9062ed54-5f71-4f96-8274-20ab5a38a9a0/volumes/kubernetes.io~projected/kube-api-access-7jzxp major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9062ed54-5f71-4f96-8274-20ab5a38a9a0/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/9062ed54-5f71-4f96-8274-20ab5a38a9a0/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/931e1236-fd6a-40f0-bd32-6e2f217eb5ff/volumes/kubernetes.io~projected/kube-api-access-5sjlf:{mountpoint:/var/lib/kubelet/pods/931e1236-fd6a-40f0-bd32-6e2f217eb5ff/volumes/kubernetes.io~projected/kube-api-access-5sjlf major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99a5bcad-5137-4a71-8130-18ab8f851b71/volumes/kubernetes.io~projected/kube-api-access-58j2j:{mountpoint:/var/lib/kubelet/pods/99a5bcad-5137-4a71-8130-18ab8f851b71/volumes/kubernetes.io~projected/kube-api-access-58j2j major:0 minor:437 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99a5bcad-5137-4a71-8130-18ab8f851b71/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/99a5bcad-5137-4a71-8130-18ab8f851b71/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:432 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99a5bcad-5137-4a71-8130-18ab8f851b71/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/99a5bcad-5137-4a71-8130-18ab8f851b71/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:436 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~projected/kube-api-access-n2v7r:{mountpoint:/var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~projected/kube-api-access-n2v7r major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a021abf7-aecb-4044-8f56-86038fd77b5b/volumes/kubernetes.io~projected/kube-api-access-87kzd:{mountpoint:/var/lib/kubelet/pods/a021abf7-aecb-4044-8f56-86038fd77b5b/volumes/kubernetes.io~projected/kube-api-access-87kzd major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aae100a6-d602-4c8b-b301-2f7a7772fe77/volumes/kubernetes.io~projected/kube-api-access-plbs2:{mountpoint:/var/lib/kubelet/pods/aae100a6-d602-4c8b-b301-2f7a7772fe77/volumes/kubernetes.io~projected/kube-api-access-plbs2 major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aae100a6-d602-4c8b-b301-2f7a7772fe77/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/aae100a6-d602-4c8b-b301-2f7a7772fe77/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:254 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aae100a6-d602-4c8b-b301-2f7a7772fe77/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/aae100a6-d602-4c8b-b301-2f7a7772fe77/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:246 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b9b6d0b0-68ba-4d89-9476-2727609a1264/volumes/kubernetes.io~projected/kube-api-access-drvtd:{mountpoint:/var/lib/kubelet/pods/b9b6d0b0-68ba-4d89-9476-2727609a1264/volumes/kubernetes.io~projected/kube-api-access-drvtd major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~projected/kube-api-access-fmlcb:{mountpoint:/var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~projected/kube-api-access-fmlcb major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:336 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:338 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:279 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/web-config/alertmanager/9 major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/web-config/config-reloader/6 major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~empty-dir/config-out major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~projected/kube-api-access-4p84j:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~projected/kube-api-access-4p84j major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~projected/tls-assets major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:279 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/config-volume major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/web-config major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4639182-9c75-4c83-81aa-e8e59d52fe6e/volumes/kubernetes.io~projected/kube-api-access-4zfjv:{mountpoint:/var/lib/kubelet/pods/d4639182-9c75-4c83-81aa-e8e59d52fe6e/volumes/kubernetes.io~projected/kube-api-access-4zfjv major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6bb8b1d-e87b-4167-a84a-92e9815f2206/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/d6bb8b1d-e87b-4167-a84a-92e9815f2206/volumes/kubernetes.io~secret/agent-certs major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e38ea962-08ee-44b8-8872-feec06d36ada/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/e38ea962-08ee-44b8-8872-feec06d36ada/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:399 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/6843330b8556784c08f02f1da778d8b5f2a71a81ca8dfd97f1e8e8c636597a8e/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/2a29c1aff78ed7b57d965247d38d22d5d9222d0484e592cae7cfa2e54a34fc3b/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/f8af1296216c4ccd4a7fb0f0f1b274cb8777f986b07f336eae47ade4d7e177b6/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/28886a3ebe066ad1752f73abffbddc46724d45fe51050bbf2c6ac45af20bfedd/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-112:{mountpoint:/var/lib/containers/storage/overlay/db62ae039f54c85b04bc6ea53fe3eda49f310d2d6b48ed788bdb87da9b61e653/merged major:0 minor:112 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/baf1a0b3ba4ad3423c641d66891632975af3892702b3b2736f43e30b7818f1d0/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/c394f46f361c748d3382bd439b6a07469aeccfca9e8b46b24c63bd9c9f7cc74d/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/d800a0b20b562ffa19687b93ae8bab2410ba395c6817fb3f6b0cfee2dc1dc9ce/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/e223eae19c8434a669f38bc32a42b289bfda3d56fee1ef396bc232b0364a7f4d/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/16ef6bc9806d733c795900e55f204d9228d7f063940f9c8dfce8c019c8272b68/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/62263bd77c1715684733fbb23ee210ce3b980d9ef5514321ddee5d01e7e380f7/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/b8c59d1c825b10c450cf8f36fc4eac2e2e0bb1df534a8f2d8c54afaba9529012/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/e753ed30092d506f0fc0436bdace63ef1d56c599c770b18997b99eb6ec4b6c02/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/90945ecb03e1fbe10ed518ac205407a8f9318e2da1b3347e9e2322d36840d258/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/74c52ccaf014f49b12ba94ddd4a259cdc0e6a5d6571e6c7ff90a2efc9beb32f4/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/1ce71dd63ec0345219459259819c3f5c353dfbe3ad7ca944daedd1eb716a5fc4/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/08bca869e4679a72ca82afa97a6e19101c1aa70bda4e6086f04754171450deb5/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/8d0e53769fc22675e2d5ed2b7bd5de0a28c61536e1c7d1bf2b545056fe9bcbc6/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/74e2befe9fa615f228eab44a22ea9b1f3962574c6ee6b89de514a08a6ccf0f8e/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/7a2177fff134ab8cebcbc6780dfb9465ca87ea447155c44dab63cfb8d70db5d8/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/478972e4d87a7893193e0315448096870f10a65733a428710522b52784bf573b/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/4a8b8c8199f7f00ea37e456963799667ec0fdbf161df643aa358082d82bfb21f/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/ecc525e2cc6035b8ce79ac54fb66837c7808bdf8339599349ca384e2f2774f28/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/5f800358007c77ffd26cec1947c3182ed236f3b1ec8cde02db185c5d42636c72/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/fb3b416cedc5be770c0ccfb888898cc0b02ec88c3204f580423ff784cf6a3f43/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/12d2b06552d83a49260f3db32fb33d4344a182a1e28ee02fefa2df505f74311e/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/c5647ce0cb375d09d0c6634d40528d25515123884d1a41327c16405589e64818/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/2a9eff1a99e4a760a9b392ae00770c2b016b38463ae8ff855eba25655f9fbee6/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/899261886d634388c3b7968165b39adaf33f3946415920da9873bd6bfcd6a47b/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/0e307ac425c055564b88da0f90c60e8a9083e538cf152251dda2f5c883da6e95/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-263:{mountpoint:/var/lib/containers/storage/overlay/7f5ea66bb8f22635a232297a265504ff18f618cc5ae8cd00108a2810c3c25bc6/merged major:0 minor:263 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/a938fb3ba58ee3ffde7e47e68a244bda779c1e028f88f43c6fe8abb54cdd0234/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/513ea2cd308c708d8dba3484158caa0a15e4cf7d3743e1fbe7e004892dc015af/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/51ac2d81a9d7803490a6c7b998961ab55547cbec5084b36a3168b21e5c701529/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/c81635110e97cffe078e700bd9beb9e1cc0196665ffae108c7309b81e5ff29f4/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/4e56f4fdfda4d82a5e045877591ef9c1b541a11280fef5b449d8ab3b29a99463/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/06a9851c5203774dce0cf95c94723da1338bbd9d353f06041d85e037712bafdd/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/e6cfbcbba6423d151fd9e4f44169450ea8af05ad3517f8e2c76a38794495b77c/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-299:{mountpoint:/var/lib/containers/storage/overlay/e37f6878210f5e0a2362748406633a622c6f1b241c322fd5a7d8112b10807a45/merged major:0 minor:299 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/e36354cb176804b7080d13238e3f25feae1f7057b3a8a019be430b4b1f4f5f4f/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/34ac1fdfe43658572d2609a2f3b6059852dc06a37129d96b1b94bce23334631b/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-319:{mountpoint:/var/lib/containers/storage/overlay/ac4cdf43e98dd8affc26b3ce2b9c94c8ed4561a612326870d77d0cd03c741bbc/merged major:0 minor:319 fsType:overlay blockSize:0} overlay_0-333:{mountpoint:/var/lib/containers/storage/overlay/3731587cb68fb97fdeeee044b468dec60204ede29d1faba96715db9528282fc1/merged major:0 minor:333 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/52e10c4290b29e19dca8afb28da20dd4543f6d62456cbc47643b2341cdfb2c07/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/4c00b9041e50d48d53debe268e687340c9d8a080a3c0f442d7bfe549de711b2f/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/582ca6f044c18ad75ce493eef9a4321378cfc8f4445f3056e55500030c6106eb/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/611f37306d76aa81c42d63fbd5a8be2a237371d5fec41fc2f4e3c9a023a6848f/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-357:{mountpoint:/var/lib/containers/storage/overlay/300ee62d6c6359758bef44587db39bc9a0576a6c88a8d0ed5bb7f4a241175b44/merged major:0 minor:357 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/5ea70e02cbad2ccd87f44faa55809ca5c9991a8dcc3bf3058d54dbd436d572d3/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-367:{mountpoint:/var/lib/containers/storage/overlay/4b5af025c2f98cd144ea2ad890f5a8ad6d6d362c22d4d1e641035822aba39d3d/merged major:0 minor:367 fsType:overlay blockSize:0} overlay_0-378:{mountpoint:/var/lib/containers/storage/overlay/e9c30c9fce2c84edc211cfd1c518899ab1f33a909fd62447959eaf425fd8f7be/merged major:0 minor:378 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/b8246bab0e427c482704c05ad5be5f997a524354bc84ee85d08aadde5a053193/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-386:{mountpoint:/var/lib/containers/storage/overlay/498f42c03dd27671afb590ece40198bd588b324755524adb435ace23bb601d2c/merged major:0 minor:386 fsType:overlay blockSize:0} overlay_0-388:{mountpoint:/var/lib/containers/storage/overlay/ec74abfcecbf461533d515027e97348b19eff9271623eed3c30c6e9f84c7bdf0/merged major:0 minor:388 fsType:overlay blockSize:0} overlay_0-394:{mountpoint:/var/lib/containers/storage/overlay/b4cdc626e31a40515ee3ddd1d6910ffbd03d965871c577d1c0e8426dfa4c684c/merged major:0 minor:394 fsType:overlay blockSize:0} overlay_0-409:{mountpoint:/var/lib/containers/storage/overlay/01a7eb4e351e73f06500411960ef0ab718796791b722df3e75fd4e833e1b7d23/merged major:0 minor:409 fsType:overlay blockSize:0} overlay_0-413:{mountpoint:/var/lib/containers/storage/overlay/d1c83107557671170a0dc81beec767999a6e4075b3485dc02ecac9d8cfb86094/merged major:0 minor:413 fsType:overlay blockSize:0} overlay_0-415:{mountpoint:/var/lib/containers/storage/overlay/066e8243fbc7f1f02f092f907e2c6771ac93cd32547d1715d0f274c882af99d8/merged major:0 minor:415 fsType:overlay blockSize:0} overlay_0-422:{mountpoint:/var/lib/containers/storage/overlay/c206638bf343ba2c247b8511ab1ef675698f50fc76e01340bfc81c401cbc86ba/merged major:0 minor:422 fsType:overlay blockSize:0} overlay_0-424:{mountpoint:/var/lib/containers/storage/overlay/ebe6a9e2cde57b16cd4fa40c4e40aa3dd5c2f9687b10329676c54513351f959d/merged major:0 minor:424 fsType:overlay blockSize:0} overlay_0-426:{mountpoint:/var/lib/containers/storage/overlay/5f9b35a1108ed1874750015b0a6da9a7b27737998b124ccac1a9e74d77a19557/merged major:0 minor:426 fsType:overlay blockSize:0} overlay_0-440:{mountpoint:/var/lib/containers/storage/overlay/8a030914291a4b65c5bbfa853eb4646d3d2bb6e078f9098185b66f7927db4947/merged major:0 minor:440 fsType:overlay blockSize:0} overlay_0-442:{mountpoint:/var/lib/containers/storage/overlay/8d68836e7d9df2771a22eca86895d1cb7a7986776f3e71d90de11284a37f6b8f/merged major:0 minor:442 fsType:overlay blockSize:0} overlay_0-444:{mountpoint:/var/lib/containers/storage/overlay/676597c7772a871f2648aa288dafac4cf97e41faa5b2b281e19fbfaba56b7581/merged major:0 minor:444 fsType:overlay blockSize:0} overlay_0-450:{mountpoint:/var/lib/containers/storage/overlay/609d64e454a6ea5085d7b38938cbd2162c62b734d939fb6733aaa5bd58994943/merged major:0 minor:450 fsType:overlay blockSize:0} overlay_0-456:{mountpoint:/var/lib/containers/storage/overlay/bc0fa6501fd74a7dad613fad698cf3475726bdfb01f4c8b90582401a28102497/merged major:0 minor:456 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/8bb9b56b2de1c6e4092c141d90e6d18f086d1c6bf05d6eabf6278d5dd5fa215d/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/a83340010a47004dc227597febc131c42864f9f3f48b5ffdf2229c049d895e6c/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/a13ca6203da8ec8812815c3835e56af2af6d6625751bf89a4fb41fb8b42c6382/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/86e45fbbc72bda91945fe16000907540c4e1e022b742a12a7e0a0f03f481fd5a/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/894452dabcc07ff48f8904eedeaca31a67d21bf920322dcd7e3c4d3d9076d79e/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/6386eb0cb2954804bd2ae6bdd64c0bf1dcd0b552c975e3f170ce43b7234b70fb/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/fd6153daf69ab9d62cacd606b864537dd9729021ed2dd71d2b8f1882f3b7bea3/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/b150e915ccbae30a405d01747d2e770e0688c155e6188dd92a6a32181bce1f13/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-93:{mountpoint:/var/lib/containers/storage/overlay/6292453a01e33e43f63175b6d4d8a66b313819e0cbbeec37409d13d57d34f6fe/merged major:0 minor:93 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/9b92c0a2c121ef5d5587c548db17e0f64c052f8e4b1988b2ea1321f468a33593/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 04 03:50:05.727046 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.726572    6751 manager.go:217] Machine: {Timestamp:2026-11-04 03:50:05.72609751 +0000 UTC m=+0.060035379 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092656 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a2a041c1785097e0809f4a0a68f59 SystemUUID:ec2a2a04-1c17-8509-7e08-09f4a0a68f59 BootID:f21578e1-8a91-401c-b9f5-12df5163f8d3 Filesystems:[{Device:/var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~projected/kube-api-access-fmlcb DeviceMajor:0 DeviceMinor:339 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-367 DeviceMajor:0 DeviceMinor:367 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-440 DeviceMajor:0 DeviceMinor:440 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:290 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/594c966f892813491d1f7cc9e017421c2dc1870030368c603635501ced8aa703/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4f25400e-aea9-43e3-b17f-0a68a76a4632/volumes/kubernetes.io~projected/kube-api-access-qsvkq DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:284 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-319 DeviceMajor:0 DeviceMinor:319 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/70d4859ab57d2dc451d265bca42cee959115e784869cfd5dd6b09473a5bdca57/userdata/shm DeviceMajor:0 DeviceMinor:331 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f276f8204b9a52b02e59f475a275f7e61cc344927976a6479b95da64e8e16292/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2fc3d49c43c6a5cdaf40e75b563deb9a951dd7032324a2304bc5dcb4f3988fc8/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-444 DeviceMajor:0 DeviceMinor:444 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3f25a6490432e4fec55af20c26b754a466bacdfec844667aec4125087850f471/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/68df7a49-2cdd-46c3-8730-9447708be784/volumes/kubernetes.io~projected/kube-api-access-nt9sb DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:287 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1c907e96-5900-4ede-b7a8-dbf6e6a092d0/volumes/kubernetes.io~projected/kube-api-access-h5f4f DeviceMajor:0 DeviceMinor:419 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-450 DeviceMajor:0 DeviceMinor:450 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a021abf7-aecb-4044-8f56-86038fd77b5b/volumes/kubernetes.io~projected/kube-api-access-87kzd DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9720f4fd3d9c516912ed18b22275262cd8dfc63cda1d7b06c7e23eb67e603b4f/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/74f52bd8-5314-4ba0-8834-81e86a91c1f6/volumes/kubernetes.io~projected/kube-api-access-sqkww DeviceMajor:0 DeviceMinor:256 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~projected/kube-api-access-4p84j DeviceMajor:0 DeviceMinor:291 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:288 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-378 DeviceMajor:0 DeviceMinor:378 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:285 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-333 DeviceMajor:0 DeviceMinor:333 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-422 DeviceMajor:0 DeviceMinor:422 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/53db76be9492a37e78f045cf9979b5cb6912ead66e9b291d45d5a8fb63c4b6ea/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3dbc6454498ef5643c21f406401cd53cea0aa791e9aad6e6d5a996202ed5d896/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:324 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:279 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-386 DeviceMajor:0 DeviceMinor:386 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aae100a6-d602-4c8b-b301-2f7a7772fe77/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:246 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:288 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-413 DeviceMajor:0 DeviceMinor:413 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:255 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0982ff5a-da06-4180-9838-6bfc5f2deba0/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/volumes/kubernetes.io~projected/kube-api-access-hv5wd DeviceMajor:0 DeviceMinor:257 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-357 DeviceMajor:0 DeviceMinor:357 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/931e1236-fd6a-40f0-bd32-6e2f217eb5ff/volumes/kubernetes.io~projected/kube-api-access-5sjlf DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d6bb8b1d-e87b-4167-a84a-92e9815f2206/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7eb7017d429d5a601074116d7db76253931352236a7d83f7277bcc72e1a25745/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0bb68137-3e53-42d0-a9fd-e6cddab3346c/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/36508a0c4387cbd148e95192f28fdfd39b34a9db94d58ac0e986eac0b68285cc/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/74f52bd8-5314-4ba0-8834-81e86a91c1f6/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:251 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/786aaf26a4bf4d0051e0e886149fd2053217ddfa20d268c7d55c33fd43434cd8/userdata/shm DeviceMajor:0 DeviceMinor:261 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-263 DeviceMajor:0 DeviceMinor:263 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-299 DeviceMajor:0 DeviceMinor:299 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/86f60625-8953-4f46-aa16-cc1d9142c4df/volumes/kubernetes.io~projected/kube-api-access-2qpr7 DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4f25400e-aea9-43e3-b17f-0a68a76a4632/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:283 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:338 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:288 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/57be963177127e5a032bdbe0b05cdb98adfb4baaa8b85e750df8443783cad5ca/userdata/shm DeviceMajor:0 DeviceMinor:411 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1c907e96-5900-4ede-b7a8-dbf6e6a092d0/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:418 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e/volumes/kubernetes.io~projected/kube-api-access-cgx6l DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b9b6d0b0-68ba-4d89-9476-2727609a1264/volumes/kubernetes.io~projected/kube-api-access-drvtd DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0bb68137-3e53-42d0-a9fd-e6cddab3346c/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/68df7a49-2cdd-46c3-8730-9447708be784/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:329 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:337 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d78443c7ec157858a80c21fdc16549054b1438e5cb7f224a565b3e2ca983053b/userdata/shm DeviceMajor:0 DeviceMinor:405 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9df2c3649bb0878ed89a04f42679d1ef8f81eb62a749d84e255daccabfeede60/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d4639182-9c75-4c83-81aa-e8e59d52fe6e/volumes/kubernetes.io~projected/kube-api-access-4zfjv DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:286 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:289 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/20b806b69a4b435d4b38f1fe2fec24efe2cd7e9cff9723239293b140b3aa3d18/userdata/shm DeviceMajor:0 DeviceMinor:376 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/99a5bcad-5137-4a71-8130-18ab8f851b71/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:436 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0982ff5a-da06-4180-9838-6bfc5f2deba0/volumes/kubernetes.io~projected/kube-api-access-vljp6 DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-112 DeviceMajor:0 DeviceMinor:112 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/74f52bd8-5314-4ba0-8834-81e86a91c1f6/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:253 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bffd53b9b2b4de941a8bd7a30e50852b1b76b07f7e444242dd71fa4ae587ee90/userdata/shm DeviceMajor:0 DeviceMinor:438 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-442 DeviceMajor:0 DeviceMinor:442 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a4d63b7b1d745fea45bfcef6030a7b315629c44a9f1861a86b5fa8a5e9870456/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9062ed54-5f71-4f96-8274-20ab5a38a9a0/volumes/kubernetes.io~projected/kube-api-access-7jzxp DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/aae100a6-d602-4c8b-b301-2f7a7772fe77/volumes/kubernetes.io~projected/kube-api-access-plbs2 DeviceMajor:0 DeviceMinor:258 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e38ea962-08ee-44b8-8872-feec06d36ada/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:399 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/45525316370e767ffde8e1e90ed74b9c697ca069f3c4f871c3ab12de7711e864/userdata/shm DeviceMajor:0 DeviceMinor:406 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-415 DeviceMajor:0 DeviceMinor:415 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-426 DeviceMajor:0 DeviceMinor:426 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:252 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/12f7f6d603150db4d1f8d7d6504361a27aa4ec238a45f6fede6d616e652ee6ee/userdata/shm DeviceMajor:0 DeviceMinor:265 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~projected/kube-api-access-n2v7r DeviceMajor:0 DeviceMinor:330 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:335 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:279 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/37e230e6-a451-475a-8b79-9d000021644b/volumes/kubernetes.io~projected/kube-api-access-bsqvw DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/df1d4698f496fdee79503bbe70e41b6f4f79c2f5b1b5a76bdeed9479f3b6a759/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1c907e96-5900-4ede-b7a8-dbf6e6a092d0/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:417 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-424 DeviceMajor:0 DeviceMinor:424 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4bbeb235-acfa-4f82-8edb-dc0727347f40/volumes/kubernetes.io~projected/kube-api-access-ggjv6 DeviceMajor:0 DeviceMinor:404 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/67f0de77c5a45d70de89ea8b34a7ba039802bafdab6a37f94a42a3b4cbc81291/userdata/shm DeviceMajor:0 DeviceMinor:259 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/37e230e6-a451-475a-8b79-9d000021644b/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:403 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/99a5bcad-5137-4a71-8130-18ab8f851b71/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:432 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0bb68137-3e53-42d0-a9fd-e6cddab3346c/volumes/kubernetes.io~projected/kube-api-access-jdbcn DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aae100a6-d602-4c8b-b301-2f7a7772fe77/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:254 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-388 DeviceMajor:0 DeviceMinor:388 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-409 DeviceMajor:0 DeviceMinor:409 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c44fc83f2a2cdda3fbedea17326a62e08b99b077a82879132f67aa13bc444078/userdata/shm DeviceMajor:0 DeviceMinor:420 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/99a5bcad-5137-4a71-8130-18ab8f851b71/volumes/kubernetes.io~projected/kube-api-access-58j2j DeviceMajor:0 DeviceMinor:437 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0982ff5a-da06-4180-9838-6bfc5f2deba0/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:288 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e433bbe80c3f57d3441236b9113fdbe7b990e68833e7f23814978187501c1f45/userdata/shm DeviceMajor:0 DeviceMinor:292 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/83f09eac-bda3-430a-958a-2a9bef3ad8e0/volumes/kubernetes.io~projected/kube-api-access-zgfvm DeviceMajor:0 DeviceMinor:372 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-93 DeviceMajor:0 DeviceMinor:93 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9062ed54-5f71-4f96-8274-20ab5a38a9a0/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9b84a44a-cb54-47a0-834b-d5794c8d59ad/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:328 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-394 DeviceMajor:0 DeviceMinor:394 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d0d74ca4f35221090e986f4163b534d866537d1f570aefd78de83725ee788a09/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd169cd6-f080-4fc4-82b4-cdb781a09640/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:336 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-456 DeviceMajor:0 DeviceMinor:456 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/31e5a426a6112b4597dd55e9d617496d1de8aeb96583b41aaccc5f81167af172/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:12f7f6d603150db MacAddress:c6:a9:8a:eb:19:59 Speed:10000 Mtu:8901} {Name:20b806b69a4b435 MacAddress:62:96:b7:67:ab:fe Speed:10000 Mtu:8901} {Name:2fc3d49c43c6a5c MacAddress:62:9b:6e:be:56:66 Speed:10000 Mtu:8901} {Name:3dbc6454498ef56 MacAddress:2e:4f:27:54:71:eb Speed:10000 Mtu:8901} {Name:45525316370e767 MacAddress:ea:16:96:5a:9d:98 Speed:10000 Mtu:8901} {Name:57be963177127e5 MacAddress:12:37:84:0f:50:8d Speed:10000 Mtu:8901} {Name:67f0de77c5a45d7 MacAddress:3a:38:23:83:44:3f Speed:10000 Mtu:8901} {Name:70d4859ab57d2dc MacAddress:92:06:44:7f:6a:25 Speed:10000 Mtu:8901} {Name:bffd53b9b2b4de9 MacAddress:e6:0c:3e:af:da:4f Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:fa:31:dc:e5 Speed:0 Mtu:9001} {Name:br-int MacAddress:26:37:ae:3d:a1:9a Speed:0 Mtu:8901} {Name:c44fc83f2a2cdda MacAddress:2e:86:5f:6f:eb:99 Speed:10000 Mtu:8901} {Name:d0d74ca4f352210 MacAddress:72:69:fe:44:27:71 Speed:10000 Mtu:8901} {Name:d78443c7ec15785 MacAddress:f6:55:9a:23:0d:b3 Speed:10000 Mtu:8901} {Name:df1d4698f496fde MacAddress:66:46:c8:01:74:45 Speed:10000 Mtu:8901} {Name:e433bbe80c3f57d MacAddress:f2:e8:93:13:34:5a Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:fa:31:dc:e5 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:3e:8f:bb:0a:a3:b7 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:02:24:53:92:a1:bf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 03:50:05.727252 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.727050    6751 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 03:50:05.727252 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.727096    6751 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 03:50:05.727497 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.727472    6751 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 03:50:05.727637 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.727498    6751 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-253.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 03:50:05.727637 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.727633    6751 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 03:50:05.727703 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.727641    6751 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 03:50:05.727703 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.727655    6751 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:50:05.727703 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.727674    6751 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 03:50:05.727783 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.727768    6751 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 03:50:05.727951 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.727942    6751 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 03:50:05.728052 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.728043    6751 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 03:50:05.728084 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.728077    6751 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 03:50:05.728115 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.728085    6751 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 03:50:05.728115 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.728103    6751 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 03:50:05.728115 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.728112    6751 kubelet.go:405] "Adding apiserver pod source"
Nov 04 03:50:05.728197 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.728120    6751 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 03:50:05.729284 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.729269    6751 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 03:50:05.729347 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.729319    6751 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:50:05.729821 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.729803    6751 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 03:50:05.743542 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.743020    6751 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 03:50:05.743542 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.743038    6751 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 03:50:05.743542 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.743193    6751 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 03:50:05.743729 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.743611    6751 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 03:50:05.743863 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.743846    6751 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 03:50:05.743992 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.743981    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 03:50:05.744028 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744000    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 03:50:05.744028 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744009    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 03:50:05.744028 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744018    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 03:50:05.744028 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744027    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 03:50:05.744147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744036    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 03:50:05.744147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744045    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 03:50:05.744147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744054    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 03:50:05.744147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744064    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 03:50:05.744147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744073    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 03:50:05.744147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744088    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 03:50:05.744147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744098    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 03:50:05.744147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744126    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 03:50:05.744147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.744136    6751 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 03:50:05.747496 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.747482    6751 server.go:1263] "Started kubelet"
Nov 04 03:50:05.747582 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.747559    6751 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 03:50:05.747670 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.747584    6751 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 03:50:05.747745 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.747669    6751 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 03:50:05.748004 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.747981    6751 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 03:50:05.748053 ip-10-0-129-253 systemd[1]: Started Kubernetes Kubelet.
Nov 04 03:50:05.748908 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.748892    6751 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 03:50:05.749638 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.749603    6751 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:50:05.752189 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.752175    6751 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 03:50:05.752277 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.752208    6751 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 03:50:05.755099 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.752343    6751 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 03:43:52 +0000 UTC" deadline="2028-04-25 07:06:45.890512491 +0000 UTC"
Nov 04 03:50:05.755099 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.752383    6751 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12915h16m40.138132395s"
Nov 04 03:50:05.755099 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.752388    6751 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 03:50:05.755099 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.752398    6751 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 03:50:05.755099 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.752496    6751 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 03:50:05.755099 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.753507    6751 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:50:05.755099 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.754459    6751 factory.go:55] Registering systemd factory
Nov 04 03:50:05.755099 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.754476    6751 factory.go:223] Registration of the systemd container factory successfully
Nov 04 03:50:05.755539 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.755186    6751 factory.go:153] Registering CRI-O factory
Nov 04 03:50:05.755539 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.755211    6751 factory.go:223] Registration of the crio container factory successfully
Nov 04 03:50:05.755829 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.755790    6751 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 03:50:05.755904 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.755830    6751 factory.go:103] Registering Raw factory
Nov 04 03:50:05.755983 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.755974    6751 manager.go:1199] Started watching for new ooms in manager
Nov 04 03:50:05.760515 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.760491    6751 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:50:05.760601 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.760518    6751 manager.go:319] Starting recovery of all containers
Nov 04 03:50:05.760931 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.760882    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-kube-api-access-4p84j" seLinuxMountContext=""
Nov 04 03:50:05.761027 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.760937    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0982ff5a-da06-4180-9838-6bfc5f2deba0" volumeName="kubernetes.io/projected/0982ff5a-da06-4180-9838-6bfc5f2deba0-kube-api-access-vljp6" seLinuxMountContext=""
Nov 04 03:50:05.761027 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.760972    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" volumeName="kubernetes.io/empty-dir/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-tmp" seLinuxMountContext=""
Nov 04 03:50:05.761027 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.760987    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" volumeName="kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-oauth-config" seLinuxMountContext=""
Nov 04 03:50:05.761027 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.760999    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="86f60625-8953-4f46-aa16-cc1d9142c4df" volumeName="kubernetes.io/projected/86f60625-8953-4f46-aa16-cc1d9142c4df-kube-api-access-2qpr7" seLinuxMountContext=""
Nov 04 03:50:05.761027 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761024    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="931e1236-fd6a-40f0-bd32-6e2f217eb5ff" volumeName="kubernetes.io/projected/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-kube-api-access-5sjlf" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761053    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aae100a6-d602-4c8b-b301-2f7a7772fe77" volumeName="kubernetes.io/empty-dir/aae100a6-d602-4c8b-b301-2f7a7772fe77-volume-directive-shadow" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761066    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd169cd6-f080-4fc4-82b4-cdb781a09640" volumeName="kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761102    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6bb8b1d-e87b-4167-a84a-92e9815f2206" volumeName="kubernetes.io/secret/d6bb8b1d-e87b-4167-a84a-92e9815f2206-agent-certs" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761115    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bb68137-3e53-42d0-a9fd-e6cddab3346c" volumeName="kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-config" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761127    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" volumeName="kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-serving-cert" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761151    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83f09eac-bda3-430a-958a-2a9bef3ad8e0" volumeName="kubernetes.io/projected/83f09eac-bda3-430a-958a-2a9bef3ad8e0-kube-api-access-zgfvm" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761168    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b9b6d0b0-68ba-4d89-9476-2727609a1264" volumeName="kubernetes.io/projected/b9b6d0b0-68ba-4d89-9476-2727609a1264-kube-api-access-drvtd" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761182    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd169cd6-f080-4fc4-82b4-cdb781a09640" volumeName="kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-serving-certs-ca-bundle" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761211    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-main-db" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761223    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761234    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bbeb235-acfa-4f82-8edb-dc0727347f40" volumeName="kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6" seLinuxMountContext=""
Nov 04 03:50:05.761261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761259    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4f25400e-aea9-43e3-b17f-0a68a76a4632" volumeName="kubernetes.io/configmap/4f25400e-aea9-43e3-b17f-0a68a76a4632-config-volume" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761272    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74f52bd8-5314-4ba0-8834-81e86a91c1f6" volumeName="kubernetes.io/empty-dir/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-textfile" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761284    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74f52bd8-5314-4ba0-8834-81e86a91c1f6" volumeName="kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-tls" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761308    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9062ed54-5f71-4f96-8274-20ab5a38a9a0" volumeName="kubernetes.io/empty-dir/9062ed54-5f71-4f96-8274-20ab5a38a9a0-data-volume" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761319    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-metrics-client-ca" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761334    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" volumeName="kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-service-ca" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761363    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4f25400e-aea9-43e3-b17f-0a68a76a4632" volumeName="kubernetes.io/projected/4f25400e-aea9-43e3-b17f-0a68a76a4632-kube-api-access-qsvkq" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761378    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4f25400e-aea9-43e3-b17f-0a68a76a4632" volumeName="kubernetes.io/secret/4f25400e-aea9-43e3-b17f-0a68a76a4632-metrics-tls" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761390    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68df7a49-2cdd-46c3-8730-9447708be784" volumeName="kubernetes.io/projected/68df7a49-2cdd-46c3-8730-9447708be784-kube-api-access-nt9sb" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761418    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74f52bd8-5314-4ba0-8834-81e86a91c1f6" volumeName="kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761432    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd169cd6-f080-4fc4-82b4-cdb781a09640" volumeName="kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-metrics-client-ca" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761449    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bb68137-3e53-42d0-a9fd-e6cddab3346c" volumeName="kubernetes.io/secret/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761474    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a5bcad-5137-4a71-8130-18ab8f851b71" volumeName="kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761499    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b9b6d0b0-68ba-4d89-9476-2727609a1264" volumeName="kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761520    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-out" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761541    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761572    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4f25400e-aea9-43e3-b17f-0a68a76a4632" volumeName="kubernetes.io/empty-dir/4f25400e-aea9-43e3-b17f-0a68a76a4632-tmp-dir" seLinuxMountContext=""
Nov 04 03:50:05.761874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761585    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9062ed54-5f71-4f96-8274-20ab5a38a9a0" volumeName="kubernetes.io/projected/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-api-access-7jzxp" seLinuxMountContext=""
Nov 04 03:50:05.762789 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.761609    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="931e1236-fd6a-40f0-bd32-6e2f217eb5ff" volumeName="kubernetes.io/configmap/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-iptables-alerter-script" seLinuxMountContext=""
Nov 04 03:50:05.762890 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762799    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b84a44a-cb54-47a0-834b-d5794c8d59ad" volumeName="kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-client-certs" seLinuxMountContext=""
Nov 04 03:50:05.762890 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762818    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b9b6d0b0-68ba-4d89-9476-2727609a1264" volumeName="kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 03:50:05.762890 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762837    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd169cd6-f080-4fc4-82b4-cdb781a09640" volumeName="kubernetes.io/projected/bd169cd6-f080-4fc4-82b4-cdb781a09640-kube-api-access-fmlcb" seLinuxMountContext=""
Nov 04 03:50:05.762890 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762848    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-tls-assets" seLinuxMountContext=""
Nov 04 03:50:05.762890 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762858    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" volumeName="kubernetes.io/projected/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-kube-api-access-h5f4f" seLinuxMountContext=""
Nov 04 03:50:05.762890 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762879    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68df7a49-2cdd-46c3-8730-9447708be784" volumeName="kubernetes.io/secret/68df7a49-2cdd-46c3-8730-9447708be784-cert" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762893    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b84a44a-cb54-47a0-834b-d5794c8d59ad" volumeName="kubernetes.io/projected/9b84a44a-cb54-47a0-834b-d5794c8d59ad-kube-api-access-n2v7r" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762919    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a021abf7-aecb-4044-8f56-86038fd77b5b" volumeName="kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-cni-binary-copy" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762939    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762949    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4639182-9c75-4c83-81aa-e8e59d52fe6e" volumeName="kubernetes.io/configmap/d4639182-9c75-4c83-81aa-e8e59d52fe6e-serviceca" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762964    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bb68137-3e53-42d0-a9fd-e6cddab3346c" volumeName="kubernetes.io/projected/0bb68137-3e53-42d0-a9fd-e6cddab3346c-kube-api-access-jdbcn" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762978    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="34bf14b5-d08b-4a4f-af25-2e24d53fdf4d" volumeName="kubernetes.io/configmap/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-metrics-client-ca" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762985    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74f52bd8-5314-4ba0-8834-81e86a91c1f6" volumeName="kubernetes.io/projected/74f52bd8-5314-4ba0-8834-81e86a91c1f6-kube-api-access-sqkww" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.762993    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74f52bd8-5314-4ba0-8834-81e86a91c1f6" volumeName="kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763013    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a5bcad-5137-4a71-8130-18ab8f851b71" volumeName="kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-oauth-config" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763020    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aae100a6-d602-4c8b-b301-2f7a7772fe77" volumeName="kubernetes.io/projected/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-api-access-plbs2" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763028    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd169cd6-f080-4fc4-82b4-cdb781a09640" volumeName="kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763049    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd169cd6-f080-4fc4-82b4-cdb781a09640" volumeName="kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763056    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" volumeName="kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-oauth-serving-cert" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763071    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37e230e6-a451-475a-8b79-9d000021644b" volumeName="kubernetes.io/projected/37e230e6-a451-475a-8b79-9d000021644b-kube-api-access-bsqvw" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763100    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74f52bd8-5314-4ba0-8834-81e86a91c1f6" volumeName="kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-metrics-client-ca" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763111    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83f09eac-bda3-430a-958a-2a9bef3ad8e0" volumeName="kubernetes.io/empty-dir/83f09eac-bda3-430a-958a-2a9bef3ad8e0-tmp" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763122    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9062ed54-5f71-4f96-8274-20ab5a38a9a0" volumeName="kubernetes.io/configmap/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763143    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a5bcad-5137-4a71-8130-18ab8f851b71" volumeName="kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-oauth-serving-cert" seLinuxMountContext=""
Nov 04 03:50:05.763168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763154    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a5bcad-5137-4a71-8130-18ab8f851b71" volumeName="kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-service-ca" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763201    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a5bcad-5137-4a71-8130-18ab8f851b71" volumeName="kubernetes.io/projected/99a5bcad-5137-4a71-8130-18ab8f851b71-kube-api-access-58j2j" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763221    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bb68137-3e53-42d0-a9fd-e6cddab3346c" volumeName="kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-env-overrides" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763234    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="34bf14b5-d08b-4a4f-af25-2e24d53fdf4d" volumeName="kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763244    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cc23bbb-cc49-4e50-a94a-df074fcd4d3e" volumeName="kubernetes.io/empty-dir/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-tmp-dir" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763270    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aae100a6-d602-4c8b-b301-2f7a7772fe77" volumeName="kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-metrics-client-ca" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763278    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aae100a6-d602-4c8b-b301-2f7a7772fe77" volumeName="kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-tls" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763284    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b9b6d0b0-68ba-4d89-9476-2727609a1264" volumeName="kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-binary-copy" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763297    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0982ff5a-da06-4180-9838-6bfc5f2deba0" volumeName="kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-tuned" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763313    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a5bcad-5137-4a71-8130-18ab8f851b71" volumeName="kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-serving-cert" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763320    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b84a44a-cb54-47a0-834b-d5794c8d59ad" volumeName="kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-metrics-server-audit-profiles" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763342    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd169cd6-f080-4fc4-82b4-cdb781a09640" volumeName="kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-federate-client-tls" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763350    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-cluster-tls-config" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763356    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-main-tls" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763372    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-volume" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763379    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e38ea962-08ee-44b8-8872-feec06d36ada" volumeName="kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763386    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cc23bbb-cc49-4e50-a94a-df074fcd4d3e" volumeName="kubernetes.io/projected/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-kube-api-access-cgx6l" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763408    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a5bcad-5137-4a71-8130-18ab8f851b71" volumeName="kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-console-config" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763423    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aae100a6-d602-4c8b-b301-2f7a7772fe77" volumeName="kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763430    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37e230e6-a451-475a-8b79-9d000021644b" volumeName="kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763444    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd169cd6-f080-4fc4-82b4-cdb781a09640" volumeName="kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-client-tls" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763451    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763457    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bb68137-3e53-42d0-a9fd-e6cddab3346c" volumeName="kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763473    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="34bf14b5-d08b-4a4f-af25-2e24d53fdf4d" volumeName="kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-tls" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763481    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b84a44a-cb54-47a0-834b-d5794c8d59ad" volumeName="kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-client-ca-bundle" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763495    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a021abf7-aecb-4044-8f56-86038fd77b5b" volumeName="kubernetes.io/projected/a021abf7-aecb-4044-8f56-86038fd77b5b-kube-api-access-87kzd" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763508    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d1b8f136-f4a2-4c04-a35a-e60e2788d381" volumeName="kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-web-config" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763523    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4639182-9c75-4c83-81aa-e8e59d52fe6e" volumeName="kubernetes.io/projected/d4639182-9c75-4c83-81aa-e8e59d52fe6e-kube-api-access-4zfjv" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763530    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" volumeName="kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-config" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763557    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="34bf14b5-d08b-4a4f-af25-2e24d53fdf4d" volumeName="kubernetes.io/projected/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-kube-api-access-hv5wd" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763565    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b84a44a-cb54-47a0-834b-d5794c8d59ad" volumeName="kubernetes.io/empty-dir/9b84a44a-cb54-47a0-834b-d5794c8d59ad-audit-log" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763571    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b84a44a-cb54-47a0-834b-d5794c8d59ad" volumeName="kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-tls" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763590    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a021abf7-aecb-4044-8f56-86038fd77b5b" volumeName="kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-daemon-config" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763605    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aae100a6-d602-4c8b-b301-2f7a7772fe77" volumeName="kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763612    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6bb8b1d-e87b-4167-a84a-92e9815f2206" volumeName="kubernetes.io/configmap/d6bb8b1d-e87b-4167-a84a-92e9815f2206-konnectivity-ca" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763644    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0982ff5a-da06-4180-9838-6bfc5f2deba0" volumeName="kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-tmp" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763656    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9062ed54-5f71-4f96-8274-20ab5a38a9a0" volumeName="kubernetes.io/secret/9062ed54-5f71-4f96-8274-20ab5a38a9a0-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763666    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a5bcad-5137-4a71-8130-18ab8f851b71" volumeName="kubernetes.io/empty-dir/99a5bcad-5137-4a71-8130-18ab8f851b71-tmp" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763683    6751 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b84a44a-cb54-47a0-834b-d5794c8d59ad" volumeName="kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763691    6751 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 03:50:05.763765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.763696    6751 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 03:50:05.770093 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.770076    6751 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 03:50:05.775996 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.775929    6751 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 03:50:05.777639 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.777586    6751 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 03:50:05.777706 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.777633    6751 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 03:50:05.777706 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.777672    6751 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 03:50:05.777799 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:05.777750    6751 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 03:50:05.785820 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.785803    6751 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 03:50:05.792891 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.792866    6751 generic.go:356] "Generic (PLEG): container finished" podID="74f52bd8-5314-4ba0-8834-81e86a91c1f6" containerID="5cb364e40c845be6c7bfc2c2288189afdb15f8c0d0fbb6701b53e0eac49b6108" exitCode=0
Nov 04 03:50:05.827519 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.827475    6751 generic.go:356] "Generic (PLEG): container finished" podID="fecc599dcc2092a8cb017231452a28ae" containerID="68d5ea0481c356ceb02c5cd997292eda75d9ce2fefcf56cf31cce9cb182ff64b" exitCode=0
Nov 04 03:50:05.835043 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.835014    6751 generic.go:356] "Generic (PLEG): container finished" podID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerID="0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103" exitCode=0
Nov 04 03:50:05.840702 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.840678    6751 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="1af84be52b05c51f2be520ae3d5d6d390ff568844a30210dd53ac49b79575c24" exitCode=0
Nov 04 03:50:05.840702 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.840700    6751 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="168a7d05ba3a5119c9b280a16a367840072a4f038cf9742c3e75cbab63dc1baf" exitCode=0
Nov 04 03:50:05.840842 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.840710    6751 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="ee49abd14e404ac2c8a70ed0ab9aba9a7e329418ed7fc78d2e3d5987dd402258" exitCode=0
Nov 04 03:50:05.840842 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.840716    6751 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="e3f72fa14f8aa14a38b2b57190838561b37722bf38930a4fa2a9899d8330f709" exitCode=0
Nov 04 03:50:05.840842 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.840721    6751 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="d14b2490a3a540cc8f6442cd0065cbe85d316e9515816d05818332ab7fbea45b" exitCode=0
Nov 04 03:50:05.840842 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.840730    6751 generic.go:356] "Generic (PLEG): container finished" podID="b9b6d0b0-68ba-4d89-9476-2727609a1264" containerID="de2383377be564c5877aa2f22b05c45a958aca62d96d2c840e2dd8a8671a6068" exitCode=0
Nov 04 03:50:05.867951 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.867909    6751 manager.go:324] Recovery completed
Nov 04 03:50:05.878587 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:05.878567    6751 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 04 03:50:05.890034 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.890015    6751 cpu_manager.go:226] "Starting" policy="none"
Nov 04 03:50:05.890034 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.890032    6751 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 03:50:05.890192 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.890050    6751 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 03:50:05.890192 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.890162    6751 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 03:50:05.890192 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.890175    6751 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 03:50:05.890192 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.890191    6751 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 03:50:05.890353 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.890196    6751 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 03:50:05.890353 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.890202    6751 policy_none.go:50] "Start"
Nov 04 03:50:05.890353 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.890211    6751 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 03:50:05.890353 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.890220    6751 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 03:50:05.891795 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.891782    6751 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 03:50:05.891795 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.891794    6751 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 03:50:05.891860 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.891801    6751 policy_none.go:44] "Start"
Nov 04 03:50:05.896770 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.896296    6751 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 03:50:05.902108 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.902087    6751 manager.go:341] "Starting Device Plugin manager"
Nov 04 03:50:05.902204 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:05.902126    6751 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 03:50:05.902204 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.902144    6751 server.go:87] "Starting device plugin registration server"
Nov 04 03:50:05.902322 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.902311    6751 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 03:50:05.902363 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.902323    6751 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 03:50:05.902457 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.902442    6751 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 03:50:05.902563 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.902532    6751 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 03:50:05.902646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.902572    6751 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 03:50:05.902646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.902579    6751 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 03:50:05.902806 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.902659    6751 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T03:50:05.902562371Z","UUID":"0cbbb2e9-21b7-46d1-bb54-be9a36413123","Handler":null,"Name":"","Endpoint":""}
Nov 04 03:50:05.903967 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.903949    6751 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 03:50:05.904051 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:05.903972    6751 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 03:50:05.904202 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:05.904183    6751 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 03:50:06.003418 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.003397    6751 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 03:50:06.005216 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.005201    6751 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 03:50:06.005270 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.005228    6751 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 03:50:06.005270 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.005238    6751 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-253.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 03:50:06.005326 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.005272    6751 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.020313 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.020294    6751 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.020375 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.020348    6751 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.078702 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.078664    6751 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"]
Nov 04 03:50:06.079319 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.079303    6751 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.079396 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.079365    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal" event={"ID":"fecc599dcc2092a8cb017231452a28ae","Type":"ContainerStarted","Data":"59f5307dfa2efa267a0eadf23992ad0689040a12657f649b458b84e292d9043a"}
Nov 04 03:50:06.079444 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.079413    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal" event={"ID":"fecc599dcc2092a8cb017231452a28ae","Type":"ContainerDied","Data":"68d5ea0481c356ceb02c5cd997292eda75d9ce2fefcf56cf31cce9cb182ff64b"}
Nov 04 03:50:06.079444 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.079432    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal" event={"ID":"fecc599dcc2092a8cb017231452a28ae","Type":"ContainerStarted","Data":"9df2c3649bb0878ed89a04f42679d1ef8f81eb62a749d84e255daccabfeede60"}
Nov 04 03:50:06.079444 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.079436    6751 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.079540 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.079450    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal" event={"ID":"f909644ddda3f20f4057bfd2523b2509","Type":"ContainerStarted","Data":"cc2a1f35b0cc4ae28fdff944c21ae5fcff47bb030d1020c10a80859ff05cf347"}
Nov 04 03:50:06.079540 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.079467    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal" event={"ID":"f909644ddda3f20f4057bfd2523b2509","Type":"ContainerStarted","Data":"a4d63b7b1d745fea45bfcef6030a7b315629c44a9f1861a86b5fa8a5e9870456"}
Nov 04 03:50:06.091214 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.091195    6751 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:50:06.091278 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:06.091230    6751 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.091873 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.091859    6751 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:50:06.091931 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:06.091891    6751 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.171235 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.171173    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.171235 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.171208    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.171235 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.171225    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f909644ddda3f20f4057bfd2523b2509-config\") pod \"kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"f909644ddda3f20f4057bfd2523b2509\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.271994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.271962    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.272087 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.272006    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.272087 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.272031    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.272087 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.272031    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f909644ddda3f20f4057bfd2523b2509-config\") pod \"kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"f909644ddda3f20f4057bfd2523b2509\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.272087 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.272064    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f909644ddda3f20f4057bfd2523b2509-config\") pod \"kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"f909644ddda3f20f4057bfd2523b2509\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.272087 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.272070    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fecc599dcc2092a8cb017231452a28ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" (UID: \"fecc599dcc2092a8cb017231452a28ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.728992 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.728966    6751 apiserver.go:52] "Watching apiserver"
Nov 04 03:50:06.744291 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.744268    6751 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 03:50:06.744884 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.744863    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5558d76459-nzh9f","openshift-ovn-kubernetes/ovnkube-node-89gb5","openshift-cluster-node-tuning-operator/tuned-95jzm","openshift-monitoring/alertmanager-main-0","openshift-console/console-579797c45f-dt5hr","openshift-insights/insights-runtime-extractor-qkmp2","openshift-monitoring/node-exporter-nxhrm","kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r","openshift-image-registry/node-ca-7jmdr","openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc","openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl","openshift-dns/node-resolver-5n7tp","openshift-multus/multus-additional-cni-plugins-vqkzw","kube-system/global-pull-secret-syncer-ksb9x","openshift-console/downloads-6dd789c8b7-mv47f","openshift-ingress-canary/ingress-canary-s4h9x","openshift-multus/multus-lw2sq","openshift-dns/dns-default-lcn4v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal","openshift-monitoring/telemeter-client-9f6fc89b6-7k47b","openshift-multus/network-metrics-daemon-hn8bx","openshift-network-diagnostics/network-check-target-g97j2","kube-system/konnectivity-agent-49hmt","openshift-network-operator/iptables-alerter-wmvzs","openshift-console/console-5c5d7fb6c9-6gqmq"]
Nov 04 03:50:06.753908 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.753862    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.753908 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.753896    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 03:50:06.754106 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.753978    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-qdtd9\""
Nov 04 03:50:06.754106 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754060    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.754106 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754058    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4ttlw\""
Nov 04 03:50:06.754235 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754121    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.754235 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754150    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-49g8t\""
Nov 04 03:50:06.754235 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754165    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.754235 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754170    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.754509 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754492    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x8rgd\""
Nov 04 03:50:06.754509 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754505    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfbg\""
Nov 04 03:50:06.754782 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754766    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m5nwx\""
Nov 04 03:50:06.754861 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754834    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 03:50:06.754929 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.754899    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 03:50:06.755391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.755351    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-p2j7s\""
Nov 04 03:50:06.755474 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.755408    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-75zvg\""
Nov 04 03:50:06.755474 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.755411    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 03:50:06.755558 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.755488    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.755685 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.755664    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 03:50:06.755825 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.755811    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tl6ng\""
Nov 04 03:50:06.755887 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.755870    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 03:50:06.756602 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.756585    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.756691 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.756586    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.756691 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.756605    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.756691 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.756661    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.756691 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.756667    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 03:50:06.756832 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.756698    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.756832 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.756756    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 03:50:06.756832 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.756811    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.756920 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.756842    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.756971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.756594    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.758119 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.758101    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 03:50:06.758214 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.758124    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 03:50:06.758214 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.758129    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 03:50:06.758214 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.758129    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-78gsl\""
Nov 04 03:50:06.758338 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.758323    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j5cz2\""
Nov 04 03:50:06.758369 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.758343    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 03:50:06.758880 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.758862    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 03:50:06.759773 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.759729    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 03:50:06.759945 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.759925    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.760014 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.759972    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.760053 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.760017    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-49k7r\""
Nov 04 03:50:06.760108 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.760095    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ccqtd\""
Nov 04 03:50:06.760487 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.760473    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-twz4j\""
Nov 04 03:50:06.763175 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763154    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.763175 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763160    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.763318 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763183    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.763318 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763191    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 03:50:06.763318 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763164    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.763318 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763173    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.763318 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763264    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 03:50:06.763318 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763271    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 04 03:50:06.763318 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763281    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 03:50:06.763318 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763273    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 03:50:06.763597 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763330    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 03:50:06.763597 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763551    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 03:50:06.763597 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.763570    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.764790 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.764774    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 03:50:06.764841 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.764804    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 03:50:06.768661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.768641    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 03:50:06.768735 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.768659    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 03:50:06.768735 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.768704    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 04 03:50:06.768735 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.768714    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 03:50:06.769598 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.769579    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 03:50:06.769724 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.769662    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 03:50:06.769724 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.769675    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 04 03:50:06.769724 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.769675    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 04 03:50:06.769823 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.769775    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 04 03:50:06.772228 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.772200    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rjs4c\""
Nov 04 03:50:06.772228 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.772228    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 03:50:06.772363 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.772231    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 04 03:50:06.772363 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.772230    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 03:50:06.772363 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.772271    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 03:50:06.772363 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.772271    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 03:50:06.772542 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.772522    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 03:50:06.773407 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.773395    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 04 03:50:06.773774 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.773755    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 03:50:06.773869 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.773853    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 03:50:06.773922 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.773867    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 04 03:50:06.773969 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.773869    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 04 03:50:06.774128 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.773916    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 04 03:50:06.774224 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774196    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 04 03:50:06.774282 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774275    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 03:50:06.774470 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774456    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 03:50:06.774470 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774470    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rh4kv\""
Nov 04 03:50:06.774648 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774522    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdbcn\" (UniqueName: \"kubernetes.io/projected/0bb68137-3e53-42d0-a9fd-e6cddab3346c-kube-api-access-jdbcn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.774694 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774670    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-system-cni-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.774747 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774707    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.774747 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774735    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:50:06.774854 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774819    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-device-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.774910 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774864    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-dbus\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:50:06.774962 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774905    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4639182-9c75-4c83-81aa-e8e59d52fe6e-host\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:50:06.775012 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774947    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zfjv\" (UniqueName: \"kubernetes.io/projected/d4639182-9c75-4c83-81aa-e8e59d52fe6e-kube-api-access-4zfjv\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:50:06.775012 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.774998    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-bin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.775111 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775051    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-tmp-dir\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:50:06.775223 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775202    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.775315 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775286    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-script-lib\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.775360 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775344    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6bb8b1d-e87b-4167-a84a-92e9815f2206-konnectivity-ca\") pod \"konnectivity-agent-49hmt\" (UID: \"d6bb8b1d-e87b-4167-a84a-92e9815f2206\") " pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:50:06.775445 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775430    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 04 03:50:06.775635 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775602    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-tmp-dir\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:50:06.775688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775575    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-binary-copy\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.775688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775665    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5cbvs\""
Nov 04 03:50:06.775780 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775726    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-etc-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.775780 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775776    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-config\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.775969 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775951    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-binary-copy\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.776023 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775956    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6bb8b1d-e87b-4167-a84a-92e9815f2206-konnectivity-ca\") pod \"konnectivity-agent-49hmt\" (UID: \"d6bb8b1d-e87b-4167-a84a-92e9815f2206\") " pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:50:06.776023 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.775999    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovn-node-metrics-cert\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.776108 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776064    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-cnibin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.776158 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776135    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-config\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.776237 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776218    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 04 03:50:06.776274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776221    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovnkube-script-lib\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.776274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776240    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-cni-binary-copy\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.776357 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776285    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bb68137-3e53-42d0-a9fd-e6cddab3346c-ovn-node-metrics-cert\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.776357 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776298    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-etc-kubernetes\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.776456 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776400    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-74vjr0sga3qj\""
Nov 04 03:50:06.776522 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776326    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-kubelet\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.776569 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776545    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-sys-fs\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.776645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776560    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mxr5f\""
Nov 04 03:50:06.776645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776583    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:50:06.776645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776613    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-run\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.776783 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776669    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-var-lib-kubelet\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.776783 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776565    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-cni-binary-copy\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.776783 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776702    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-kubelet\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.776783 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776754    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:50:06.776974 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776798    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-env-overrides\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.776974 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776849    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-lib-modules\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.776974 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776875    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysconfig\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.776974 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776905    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.776974 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776932    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-host\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.777186 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776989    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bb68137-3e53-42d0-a9fd-e6cddab3346c-env-overrides\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.777186 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.776691    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 03:50:06.777186 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777013    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsqvw\" (UniqueName: \"kubernetes.io/projected/37e230e6-a451-475a-8b79-9d000021644b-kube-api-access-bsqvw\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:50:06.777307 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777291    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38ea962-08ee-44b8-8872-feec06d36ada-original-pull-secret\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:50:06.777431 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777409    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-netns\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.777584 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777544    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-systemd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.777659 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777591    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37e230e6-a451-475a-8b79-9d000021644b-metrics-certs\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:50:06.777659 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777643    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-var-lib-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.777747 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777693    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 04 03:50:06.777747 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777692    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-log-socket\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.777747 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777728    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n475p\""
Nov 04 03:50:06.777880 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777761    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.777880 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777842    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-pkj8n\""
Nov 04 03:50:06.777971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777844    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-ovn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.777971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777934    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-bin\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.778060 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.777955    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-p8xwh\""
Nov 04 03:50:06.778060 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778019    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.778060 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778029    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 03:50:06.778199 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778080    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-socket-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.778199 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778152    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qpr7\" (UniqueName: \"kubernetes.io/projected/86f60625-8953-4f46-aa16-cc1d9142c4df-kube-api-access-2qpr7\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.778199 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778159    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 04 03:50:06.778348 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778298    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5s97h\""
Nov 04 03:50:06.778348 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778326    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fg6jl\""
Nov 04 03:50:06.778434 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778300    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-modprobe-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.778480 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778445    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4639182-9c75-4c83-81aa-e8e59d52fe6e-serviceca\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:50:06.778534 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778487    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-slash\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.778534 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778518    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.778647 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778548    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.778647 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778574    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-systemd\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.778647 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778598    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-tmp\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.778795 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778648    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.778795 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778684    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-socket-dir-parent\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.778795 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778714    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-multus\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.778795 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778743    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-conf\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.778795 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778764    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 03:50:06.778795 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778768    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-netd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.779067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778796    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.779067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778831    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6bb8b1d-e87b-4167-a84a-92e9815f2206-agent-certs\") pod \"konnectivity-agent-49hmt\" (UID: \"d6bb8b1d-e87b-4167-a84a-92e9815f2206\") " pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:50:06.779067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778862    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-kubelet-config\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:50:06.779067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778890    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-conf-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.779067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778915    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-multus-certs\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.779067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778949    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87kzd\" (UniqueName: \"kubernetes.io/projected/a021abf7-aecb-4044-8f56-86038fd77b5b-kube-api-access-87kzd\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.779067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.778987    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-systemd-units\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.779067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779017    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-node-log\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.779067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779047    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drvtd\" (UniqueName: \"kubernetes.io/projected/b9b6d0b0-68ba-4d89-9476-2727609a1264-kube-api-access-drvtd\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.779485 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779074    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.779485 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779111    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-registration-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.779485 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779156    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-k8s-cni-cncf-io\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.779485 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779192    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-netns\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.779485 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779228    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-hostroot\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.779485 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779286    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-host-slash\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:50:06.779485 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779311    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4639182-9c75-4c83-81aa-e8e59d52fe6e-serviceca\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:50:06.779485 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779397    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sjlf\" (UniqueName: \"kubernetes.io/projected/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-kube-api-access-5sjlf\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:50:06.779485 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779430    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vljp6\" (UniqueName: \"kubernetes.io/projected/0982ff5a-da06-4180-9838-6bfc5f2deba0-kube-api-access-vljp6\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.779973 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779503    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-hosts-file\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:50:06.779973 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779452    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-tmp\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.779973 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779572    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6bb8b1d-e87b-4167-a84a-92e9815f2206-agent-certs\") pod \"konnectivity-agent-49hmt\" (UID: \"d6bb8b1d-e87b-4167-a84a-92e9815f2206\") " pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:50:06.779973 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779853    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-sys\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.779973 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.779928    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-tuned\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.780205 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780051    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-system-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.780205 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780117    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-daemon-config\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.780205 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780122    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-tuned\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.780764 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780743    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-kubernetes\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.780839 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780787    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-cnibin\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.780839 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780812    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.780839 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780818    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 04 03:50:06.780937 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780838    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgx6l\" (UniqueName: \"kubernetes.io/projected/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-kube-api-access-cgx6l\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:50:06.780937 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780897    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-iptables-alerter-script\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:50:06.780937 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780900    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 04 03:50:06.781031 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.780956    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-daemon-config\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.781067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.781052    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-iptables-alerter-script\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:50:06.781423 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.781152    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9b6d0b0-68ba-4d89-9476-2727609a1264-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.791246 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.791222    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggjv6\" (UniqueName: \"kubernetes.io/projected/4bbeb235-acfa-4f82-8edb-dc0727347f40-kube-api-access-ggjv6\") pod \"network-check-target-g97j2\" (UID: \"4bbeb235-acfa-4f82-8edb-dc0727347f40\") " pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:50:06.791565 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.791538    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdbcn\" (UniqueName: \"kubernetes.io/projected/0bb68137-3e53-42d0-a9fd-e6cddab3346c-kube-api-access-jdbcn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.791784 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.791762    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsqvw\" (UniqueName: \"kubernetes.io/projected/37e230e6-a451-475a-8b79-9d000021644b-kube-api-access-bsqvw\") pod \"network-metrics-daemon-hn8bx\" (UID: \"37e230e6-a451-475a-8b79-9d000021644b\") " pod="openshift-multus/network-metrics-daemon-hn8bx"
Nov 04 03:50:06.791970 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.791952    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sjlf\" (UniqueName: \"kubernetes.io/projected/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-kube-api-access-5sjlf\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:50:06.792166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.792148    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgx6l\" (UniqueName: \"kubernetes.io/projected/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-kube-api-access-cgx6l\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:50:06.796527 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.796508    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zfjv\" (UniqueName: \"kubernetes.io/projected/d4639182-9c75-4c83-81aa-e8e59d52fe6e-kube-api-access-4zfjv\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:50:06.797355 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.797329    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vljp6\" (UniqueName: \"kubernetes.io/projected/0982ff5a-da06-4180-9838-6bfc5f2deba0-kube-api-access-vljp6\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.797434 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.797366    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87kzd\" (UniqueName: \"kubernetes.io/projected/a021abf7-aecb-4044-8f56-86038fd77b5b-kube-api-access-87kzd\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.797901 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.797880    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qpr7\" (UniqueName: \"kubernetes.io/projected/86f60625-8953-4f46-aa16-cc1d9142c4df-kube-api-access-2qpr7\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.798921 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.798901    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drvtd\" (UniqueName: \"kubernetes.io/projected/b9b6d0b0-68ba-4d89-9476-2727609a1264-kube-api-access-drvtd\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.845507 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.845484    6751 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.845683 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.845629    6751 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.854403 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.854382    6751 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 03:50:06.858324 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.858303    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5d7fb6c9-6gqmq"]
Nov 04 03:50:06.858569 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:06.858546    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[console-config console-oauth-config console-serving-cert kube-api-access-h5f4f oauth-serving-cert service-ca tmp], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-console/console-5c5d7fb6c9-6gqmq" podUID="1c907e96-5900-4ede-b7a8-dbf6e6a092d0"
Nov 04 03:50:06.858794 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.858781    6751 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:50:06.858846 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:06.858822    6751 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.859196 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.859173    6751 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 03:50:06.859299 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:06.859251    6751 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal"
Nov 04 03:50:06.881334 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881308    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-client-certs\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.881439 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881336    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-etc-kubernetes\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.881439 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881351    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.881439 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881375    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plbs2\" (UniqueName: \"kubernetes.io/projected/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-api-access-plbs2\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.881439 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881411    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9062ed54-5f71-4f96-8274-20ab5a38a9a0-data-volume\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.881439 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881425    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-kubelet\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881436    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-etc-kubernetes\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881442    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-sys-fs\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881487    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-sys-fs\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881501    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-web-config\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881528    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f25400e-aea9-43e3-b17f-0a68a76a4632-tmp-dir\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881548    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-kubelet\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881553    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-oauth-serving-cert\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881592    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881592    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9062ed54-5f71-4f96-8274-20ab5a38a9a0-data-volume\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881641    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-client-certs\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881667    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-volume\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881675    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f25400e-aea9-43e3-b17f-0a68a76a4632-tmp-dir\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881708    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83f09eac-bda3-430a-958a-2a9bef3ad8e0-tmp\") pod \"downloads-6dd789c8b7-mv47f\" (UID: \"83f09eac-bda3-430a-958a-2a9bef3ad8e0\") " pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:50:06.881719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881729    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-web-config\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881734    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f25400e-aea9-43e3-b17f-0a68a76a4632-metrics-tls\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881762    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-service-ca\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881769    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-oauth-serving-cert\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881784    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-host\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881820    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83f09eac-bda3-430a-958a-2a9bef3ad8e0-tmp\") pod \"downloads-6dd789c8b7-mv47f\" (UID: \"83f09eac-bda3-430a-958a-2a9bef3ad8e0\") " pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881853    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-host\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881807    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-netns\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881876    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-netns\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881893    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-systemd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881891    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-volume\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881926    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-log-socket\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881951    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881956    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-log-socket\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881926    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-systemd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881958    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-service-ca\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881977    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881992    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f25400e-aea9-43e3-b17f-0a68a76a4632-metrics-tls\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.881999    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-ovn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882014    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-socket-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882019    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882040    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-ovn\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882049    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-socket-dir-parent\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882066    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-socket-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882142    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882155    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882176    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-socket-dir-parent\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882219    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882304    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882331    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882343    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.882333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882353    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-slash\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882378    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882411    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882455    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-slash\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882460    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-conf\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882477    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882486    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882477    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882491    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882571    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-netd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882601    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882610    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-netd\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882576    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-conf\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882647    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-conf-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882678    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882692    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-etc-selinux\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882685    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-conf-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882718    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-metrics-server-audit-profiles\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882744    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-client-ca-bundle\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882766    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-node-log\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882786    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882790    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882827    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-node-log\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882837    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-registration-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882841    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-run-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882847    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882864    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-k8s-cni-cncf-io\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882882    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-registration-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882890    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-hostroot\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882903    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-k8s-cni-cncf-io\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882905    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-host-slash\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882929    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/931e1236-fd6a-40f0-bd32-6e2f217eb5ff-host-slash\") pod \"iptables-alerter-wmvzs\" (UID: \"931e1236-fd6a-40f0-bd32-6e2f217eb5ff\") " pod="openshift-network-operator/iptables-alerter-wmvzs"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882953    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882965    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-hostroot\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882983    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-client-ca-bundle\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.882982    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883012    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-root\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883027    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-metrics-server-audit-profiles\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883036    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-hosts-file\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883082    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883099    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-federate-client-tls\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883129    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2v7r\" (UniqueName: \"kubernetes.io/projected/9b84a44a-cb54-47a0-834b-d5794c8d59ad-kube-api-access-n2v7r\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883153    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-tls\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883154    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883184    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc23bbb-cc49-4e50-a94a-df074fcd4d3e-hosts-file\") pod \"node-resolver-5n7tp\" (UID: \"4cc23bbb-cc49-4e50-a94a-df074fcd4d3e\") " pod="openshift-dns/node-resolver-5n7tp"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883199    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-system-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883202    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883218    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883245    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-trusted-ca-bundle\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883270    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-kubernetes\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883274    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883319    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-kubernetes\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883335    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-system-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883358    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-federate-client-tls\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883358    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-cnibin\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883386    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-cnibin\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883426    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883433    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-tls\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883436    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-config\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883483    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-device-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883509    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt9sb\" (UniqueName: \"kubernetes.io/projected/68df7a49-2cdd-46c3-8730-9447708be784-kube-api-access-nt9sb\") pod \"ingress-canary-s4h9x\" (UID: \"68df7a49-2cdd-46c3-8730-9447708be784\") " pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883539    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883546    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-trusted-ca-bundle\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883559    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86f60625-8953-4f46-aa16-cc1d9142c4df-device-dir\") pod \"aws-ebs-csi-driver-node-nqf9r\" (UID: \"86f60625-8953-4f46-aa16-cc1d9142c4df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nqf9r"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883571    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-bin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883578    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-config\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883590    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883601    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-bin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883607    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883705    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.883661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883711    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883741    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv5wd\" (UniqueName: \"kubernetes.io/projected/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-kube-api-access-hv5wd\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883757    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5f4f\" (UniqueName: \"kubernetes.io/projected/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-kube-api-access-h5f4f\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883764    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-secret-telemeter-client\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883780    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883800    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883901    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-sys\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883906    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aae100a6-d602-4c8b-b301-2f7a7772fe77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883940    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883951    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqkww\" (UniqueName: \"kubernetes.io/projected/74f52bd8-5314-4ba0-8834-81e86a91c1f6-kube-api-access-sqkww\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.883988    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-etc-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884004    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-cnibin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884016    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-etc-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884020    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgfvm\" (UniqueName: \"kubernetes.io/projected/83f09eac-bda3-430a-958a-2a9bef3ad8e0-kube-api-access-zgfvm\") pod \"downloads-6dd789c8b7-mv47f\" (UID: \"83f09eac-bda3-430a-958a-2a9bef3ad8e0\") " pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884065    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58j2j\" (UniqueName: \"kubernetes.io/projected/99a5bcad-5137-4a71-8130-18ab8f851b71-kube-api-access-58j2j\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884088    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-run\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884097    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-cnibin\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884113    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-var-lib-kubelet\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884136    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-kubelet\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884151    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f25400e-aea9-43e3-b17f-0a68a76a4632-config-volume\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884166    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-lib-modules\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884186    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884196    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-kubelet\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884163    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-run\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884237    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9b84a44a-cb54-47a0-834b-d5794c8d59ad-audit-log\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884249    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-var-lib-kubelet\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884268    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-lib-modules\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884350    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysconfig\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884387    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b84a44a-cb54-47a0-834b-d5794c8d59ad-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884391    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9b84a44a-cb54-47a0-834b-d5794c8d59ad-audit-log\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884405    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f25400e-aea9-43e3-b17f-0a68a76a4632-config-volume\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884416    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884433    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-var-lib-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884439    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysconfig\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884460    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-modprobe-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884466    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-var-lib-openvswitch\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884489    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-sysctl-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884491    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-multus-certs\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884507    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-multus-certs\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884513    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-bin\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884529    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884533    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-modprobe-d\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884532    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-cni-bin\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884545    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884551    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884573    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-wtmp\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884575    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-multus-cni-dir\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884591    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884609    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884649    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-systemd\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884687    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-multus\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884737    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-host-run-ovn-kubernetes\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884742    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-out\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884749    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-var-lib-cni-multus\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884772    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-etc-systemd\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884764    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884821    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-oauth-config\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884835    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-out\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884881    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-kubelet-config\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884908    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsvkq\" (UniqueName: \"kubernetes.io/projected/4f25400e-aea9-43e3-b17f-0a68a76a4632-kube-api-access-qsvkq\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884917    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884932    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-serving-cert\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884880    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884957    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-systemd-units\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.884922    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-kubelet-config\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885002    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-oauth-config\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885013    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-client-tls\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885040    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-metrics-client-ca\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885043    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bb68137-3e53-42d0-a9fd-e6cddab3346c-systemd-units\") pod \"ovnkube-node-89gb5\" (UID: \"0bb68137-3e53-42d0-a9fd-e6cddab3346c\") " pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885065    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-console-config\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885087    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-serving-cert\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885088    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-oauth-serving-cert\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885117    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-metrics-client-ca\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885137    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-netns\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885155    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-serving-certs-ca-bundle\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885170    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-service-ca\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885188    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a5bcad-5137-4a71-8130-18ab8f851b71-tmp\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885215    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/aae100a6-d602-4c8b-b301-2f7a7772fe77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885220    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a021abf7-aecb-4044-8f56-86038fd77b5b-host-run-netns\") pod \"multus-lw2sq\" (UID: \"a021abf7-aecb-4044-8f56-86038fd77b5b\") " pod="openshift-multus/multus-lw2sq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885239    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9062ed54-5f71-4f96-8274-20ab5a38a9a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885260    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-metrics-client-ca\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885267    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885266    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-client-tls\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885274    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-oauth-serving-cert\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885274    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-console-config\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885402    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a5bcad-5137-4a71-8130-18ab8f851b71-tmp\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885406    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-service-ca\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885412    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74f52bd8-5314-4ba0-8834-81e86a91c1f6-metrics-client-ca\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885421    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885453    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-sys\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885481    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9062ed54-5f71-4f96-8274-20ab5a38a9a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885501    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885519    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68df7a49-2cdd-46c3-8730-9447708be784-cert\") pod \"ingress-canary-s4h9x\" (UID: \"68df7a49-2cdd-46c3-8730-9447708be784\") " pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885526    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-serving-certs-ca-bundle\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885529    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0982ff5a-da06-4180-9838-6bfc5f2deba0-sys\") pod \"tuned-95jzm\" (UID: \"0982ff5a-da06-4180-9838-6bfc5f2deba0\") " pod="openshift-cluster-node-tuning-operator/tuned-95jzm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885534    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmlcb\" (UniqueName: \"kubernetes.io/projected/bd169cd6-f080-4fc4-82b4-cdb781a09640-kube-api-access-fmlcb\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885538    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/aae100a6-d602-4c8b-b301-2f7a7772fe77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885572    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-serving-cert\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885600    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-system-cni-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885661    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-dbus\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885678    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885683    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4639182-9c75-4c83-81aa-e8e59d52fe6e-host\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885706    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68df7a49-2cdd-46c3-8730-9447708be784-cert\") pod \"ingress-canary-s4h9x\" (UID: \"68df7a49-2cdd-46c3-8730-9447708be784\") " pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885716    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p84j\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-kube-api-access-4p84j\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885721    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4639182-9c75-4c83-81aa-e8e59d52fe6e-host\") pod \"node-ca-7jmdr\" (UID: \"d4639182-9c75-4c83-81aa-e8e59d52fe6e\") " pod="openshift-image-registry/node-ca-7jmdr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885749    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9b6d0b0-68ba-4d89-9476-2727609a1264-system-cni-dir\") pod \"multus-additional-cni-plugins-vqkzw\" (UID: \"b9b6d0b0-68ba-4d89-9476-2727609a1264\") " pod="openshift-multus/multus-additional-cni-plugins-vqkzw"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885766    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38ea962-08ee-44b8-8872-feec06d36ada-dbus\") pod \"global-pull-secret-syncer-ksb9x\" (UID: \"e38ea962-08ee-44b8-8872-feec06d36ada\") " pod="kube-system/global-pull-secret-syncer-ksb9x"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885816    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-oauth-config\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885833    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-tmp\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885829    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-serving-cert\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885851    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885876    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9062ed54-5f71-4f96-8274-20ab5a38a9a0-crio-socket\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885902    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jzxp\" (UniqueName: \"kubernetes.io/projected/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-api-access-7jzxp\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885909    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-tmp\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885937    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-textfile\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885950    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-oauth-config\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.885993    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-textfile\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.886004    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-tls\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.886103    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd169cd6-f080-4fc4-82b4-cdb781a09640-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.886503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.886168    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9b84a44a-cb54-47a0-834b-d5794c8d59ad-secret-metrics-server-tls\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.896685 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.896665    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2v7r\" (UniqueName: \"kubernetes.io/projected/9b84a44a-cb54-47a0-834b-d5794c8d59ad-kube-api-access-n2v7r\") pod \"metrics-server-5558d76459-nzh9f\" (UID: \"9b84a44a-cb54-47a0-834b-d5794c8d59ad\") " pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:06.896775 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.896716    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plbs2\" (UniqueName: \"kubernetes.io/projected/aae100a6-d602-4c8b-b301-2f7a7772fe77-kube-api-access-plbs2\") pod \"kube-state-metrics-5dc8d7d89b-bgjtc\" (UID: \"aae100a6-d602-4c8b-b301-2f7a7772fe77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-bgjtc"
Nov 04 03:50:06.901767 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.901710    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv5wd\" (UniqueName: \"kubernetes.io/projected/34bf14b5-d08b-4a4f-af25-2e24d53fdf4d-kube-api-access-hv5wd\") pod \"openshift-state-metrics-7bd7489957-nmfcl\" (UID: \"34bf14b5-d08b-4a4f-af25-2e24d53fdf4d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nmfcl"
Nov 04 03:50:06.901767 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.901739    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58j2j\" (UniqueName: \"kubernetes.io/projected/99a5bcad-5137-4a71-8130-18ab8f851b71-kube-api-access-58j2j\") pod \"console-579797c45f-dt5hr\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") " pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:06.901767 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.901743    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgfvm\" (UniqueName: \"kubernetes.io/projected/83f09eac-bda3-430a-958a-2a9bef3ad8e0-kube-api-access-zgfvm\") pod \"downloads-6dd789c8b7-mv47f\" (UID: \"83f09eac-bda3-430a-958a-2a9bef3ad8e0\") " pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:50:06.902122 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.902099    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt9sb\" (UniqueName: \"kubernetes.io/projected/68df7a49-2cdd-46c3-8730-9447708be784-kube-api-access-nt9sb\") pod \"ingress-canary-s4h9x\" (UID: \"68df7a49-2cdd-46c3-8730-9447708be784\") " pod="openshift-ingress-canary/ingress-canary-s4h9x"
Nov 04 03:50:06.902232 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.902106    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5f4f\" (UniqueName: \"kubernetes.io/projected/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-kube-api-access-h5f4f\") pod \"console-5c5d7fb6c9-6gqmq\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") " pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:06.902274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.902256    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p84j\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-kube-api-access-4p84j\") pod \"alertmanager-main-0\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:06.902468 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.902450    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmlcb\" (UniqueName: \"kubernetes.io/projected/bd169cd6-f080-4fc4-82b4-cdb781a09640-kube-api-access-fmlcb\") pod \"telemeter-client-9f6fc89b6-7k47b\" (UID: \"bd169cd6-f080-4fc4-82b4-cdb781a09640\") " pod="openshift-monitoring/telemeter-client-9f6fc89b6-7k47b"
Nov 04 03:50:06.906789 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.906773    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsvkq\" (UniqueName: \"kubernetes.io/projected/4f25400e-aea9-43e3-b17f-0a68a76a4632-kube-api-access-qsvkq\") pod \"dns-default-lcn4v\" (UID: \"4f25400e-aea9-43e3-b17f-0a68a76a4632\") " pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:50:06.907688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.907673    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqkww\" (UniqueName: \"kubernetes.io/projected/74f52bd8-5314-4ba0-8834-81e86a91c1f6-kube-api-access-sqkww\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.907740 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.907707    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jzxp\" (UniqueName: \"kubernetes.io/projected/9062ed54-5f71-4f96-8274-20ab5a38a9a0-kube-api-access-7jzxp\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.986545 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.986488    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-wtmp\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.986545 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.986525    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9062ed54-5f71-4f96-8274-20ab5a38a9a0-crio-socket\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.986734 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.986555    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-root\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.986734 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.986613    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-node-exporter-wtmp\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.986802 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.986587    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-sys\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.990262 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.986648    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9062ed54-5f71-4f96-8274-20ab5a38a9a0-crio-socket\") pod \"insights-runtime-extractor-qkmp2\" (UID: \"9062ed54-5f71-4f96-8274-20ab5a38a9a0\") " pod="openshift-insights/insights-runtime-extractor-qkmp2"
Nov 04 03:50:06.990262 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.986688    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-sys\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:06.990262 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:06.986646    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74f52bd8-5314-4ba0-8834-81e86a91c1f6-root\") pod \"node-exporter-nxhrm\" (UID: \"74f52bd8-5314-4ba0-8834-81e86a91c1f6\") " pod="openshift-monitoring/node-exporter-nxhrm"
Nov 04 03:50:07.061232 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:07.061211    6751 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:50:08.982520 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:08.982487    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lcn4v"
Nov 04 03:50:09.932820 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:09.932769    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:10.137364 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:10.137328    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:11.248117 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:11.248082    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:50:11.501181 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:11.501111    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:13.364424 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:13.364344    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:13.804063 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:13.803991    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:13.844519 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:13.844490    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:14.061067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:14.060970    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:14.540920 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:14.540895    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:14.541323 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:14.541114    6751 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 03:50:15.528309 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:15.528271    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:15.730880 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:15.730840    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:20.993444 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:20.993416    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-49hmt"
Nov 04 03:50:21.091096 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:21.091060    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:50:21.335811 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:21.335735    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:21.506094 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:21.506065    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:21.509600 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:21.509585    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:50:21.538072 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:21.538042    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:25.989273 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:25.989245    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5558d76459-nzh9f"
Nov 04 03:50:26.915840 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:26.915801    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:27.121283 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:27.121232    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:27.438175 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:27.438146    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89gb5"
Nov 04 03:50:28.265590 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.265554    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-795f6cf888-8798b"]
Nov 04 03:50:28.268601 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.268583    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.282788 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.282743    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-795f6cf888-8798b"]
Nov 04 03:50:28.334925 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.334897    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:28.364135 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.364108    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-oauth-serving-cert\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.364248 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.364161    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-trusted-ca-bundle\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.364295 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.364281    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-tmp\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.364346 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.364316    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx4x7\" (UniqueName: \"kubernetes.io/projected/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-kube-api-access-jx4x7\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.364346 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.364341    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-console-config\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.364487 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.364470    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-console-oauth-config\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.364551 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.364512    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-console-serving-cert\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.364631 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.364598    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-service-ca\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.465430 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.465408    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-service-ca\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.465544 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.465520    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-oauth-serving-cert\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.465613 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.465578    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-trusted-ca-bundle\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.465698 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.465667    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-tmp\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.465751 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.465709    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx4x7\" (UniqueName: \"kubernetes.io/projected/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-kube-api-access-jx4x7\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.465808 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.465736    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-console-config\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.465808 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.465789    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-console-oauth-config\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.465910 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.465825    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-console-serving-cert\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.466050 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.466028    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-tmp\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.466298 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.466278    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-service-ca\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.466376 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.466302    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-oauth-serving-cert\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.466376 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.466319    6751 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 03:50:28.466655 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.466614    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-console-config\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.466764 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.466659    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-trusted-ca-bundle\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.468730 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.468712    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-console-serving-cert\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.468817 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.468716    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-console-oauth-config\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.478394 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.478373    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx4x7\" (UniqueName: \"kubernetes.io/projected/2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a-kube-api-access-jx4x7\") pod \"console-795f6cf888-8798b\" (UID: \"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a\") " pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.536939 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.536871    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:28.579311 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.579279    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:28.855102 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.855066    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-795f6cf888-8798b"]
Nov 04 03:50:28.856733 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:28.856692    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc79c0c_7d55_46f7_a19e_09f92b6c8f8a.slice/crio-ff265f6363f0439eca6347108f3f2f9fe5b95929408c503b1db2a217255ad252 WatchSource:0}: Error finding container ff265f6363f0439eca6347108f3f2f9fe5b95929408c503b1db2a217255ad252: Status 404 returned error can't find the container with id ff265f6363f0439eca6347108f3f2f9fe5b95929408c503b1db2a217255ad252
Nov 04 03:50:28.932463 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.932431    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795f6cf888-8798b" event={"ID":"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a","Type":"ContainerStarted","Data":"82db0a665ebc6f9680007d8021fc840dafd20c419a45c66d3bf0a62e5cb587b3"}
Nov 04 03:50:28.932594 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.932487    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795f6cf888-8798b" event={"ID":"2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a","Type":"ContainerStarted","Data":"ff265f6363f0439eca6347108f3f2f9fe5b95929408c503b1db2a217255ad252"}
Nov 04 03:50:28.955604 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:28.955553    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-795f6cf888-8798b" podStartSLOduration=0.95553945 podStartE2EDuration="955.53945ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:50:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:50:28.955365565 +0000 UTC m=+23.289303440" watchObservedRunningTime="2026-11-04 03:50:28.95553945 +0000 UTC m=+23.289477327"
Nov 04 03:50:30.329142 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:30.329071    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:30.540007 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:30.539974    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:31.100175 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.100142    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:50:31.100537 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.100505    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="alertmanager" containerID="cri-o://dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1" gracePeriod=120
Nov 04 03:50:31.100603 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.100553    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy-web" containerID="cri-o://f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f" gracePeriod=120
Nov 04 03:50:31.100603 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.100555    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy-metric" containerID="cri-o://fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078" gracePeriod=120
Nov 04 03:50:31.100732 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.100616    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy" containerID="cri-o://98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61" gracePeriod=120
Nov 04 03:50:31.100732 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.100639    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="config-reloader" containerID="cri-o://a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1" gracePeriod=120
Nov 04 03:50:31.100732 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.100707    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="prom-label-proxy" containerID="cri-o://ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2" gracePeriod=120
Nov 04 03:50:31.949047 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.949005    6751 generic.go:356] "Generic (PLEG): container finished" podID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerID="ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2" exitCode=0
Nov 04 03:50:31.949047 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.949039    6751 generic.go:356] "Generic (PLEG): container finished" podID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerID="fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078" exitCode=0
Nov 04 03:50:31.949047 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.949046    6751 generic.go:356] "Generic (PLEG): container finished" podID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerID="98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61" exitCode=0
Nov 04 03:50:31.949047 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.949051    6751 generic.go:356] "Generic (PLEG): container finished" podID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerID="a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1" exitCode=0
Nov 04 03:50:31.949569 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.949058    6751 generic.go:356] "Generic (PLEG): container finished" podID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerID="dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1" exitCode=0
Nov 04 03:50:31.949569 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.949078    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerDied","Data":"ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2"}
Nov 04 03:50:31.949569 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.949147    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerDied","Data":"fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078"}
Nov 04 03:50:31.949569 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.949176    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerDied","Data":"98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61"}
Nov 04 03:50:31.949569 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.949201    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerDied","Data":"a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1"}
Nov 04 03:50:31.949569 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:31.949225    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerDied","Data":"dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1"}
Nov 04 03:50:32.335426 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.335389    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:32.539537 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.539455    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:32.858896 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.858870    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:32.870071 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.869962    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5c5d7fb6c9-6gqmq" podUID="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" containerName="console" containerID="cri-o://57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192" gracePeriod=15
Nov 04 03:50:32.893246 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.893216    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:32.922289 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922267    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922463 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922299    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-trusted-ca-bundle\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922463 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922317    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-web-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-web-config\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922463 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922344    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-out\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922463 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922376    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-web\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922463 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922411    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-kube-api-access-4p84j\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-kube-api-access-4p84j\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922463 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922444    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-metrics-client-ca\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922806 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922474    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-cluster-tls-config\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922806 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922516    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-volume\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-volume\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922806 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922542    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-tls-assets\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922806 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922584    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922806 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922684    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-main-db\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.922806 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922732    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-main-tls\") pod \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\" (UID: \"d1b8f136-f4a2-4c04-a35a-e60e2788d381\") "
Nov 04 03:50:32.923101 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922823    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-trusted-ca-bundle" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:32.923101 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.922997    6751 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:32.923880 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.923537    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-metrics-client-ca" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:32.923880 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.923600    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-main-db" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:32.925820 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.925792    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-main-tls" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:32.926984 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.926955    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-web" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:32.927120 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.926990    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-out" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:32.927120 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.927028    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-tls-assets" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:32.927290 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.927032    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-metric" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:32.928272 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.928243    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:32.928364 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.928304    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-kube-api-access-4p84j" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "kube-api-access-4p84j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:32.928415 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.928392    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-volume" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:32.932795 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.932770    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-cluster-tls-config" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:32.939361 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.939335    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-web-config" pod "d1b8f136-f4a2-4c04-a35a-e60e2788d381" (UID: "d1b8f136-f4a2-4c04-a35a-e60e2788d381"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:32.958261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.958235    6751 generic.go:356] "Generic (PLEG): container finished" podID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerID="f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f" exitCode=0
Nov 04 03:50:32.958675 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.958280    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerDied","Data":"f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f"}
Nov 04 03:50:32.958675 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.958347    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:32.958675 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.958359    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b8f136-f4a2-4c04-a35a-e60e2788d381","Type":"ContainerDied","Data":"e433bbe80c3f57d3441236b9113fdbe7b990e68833e7f23814978187501c1f45"}
Nov 04 03:50:32.958675 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.958396    6751 scope.go:122] "RemoveContainer" containerID="ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2"
Nov 04 03:50:32.982140 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.982007    6751 scope.go:122] "RemoveContainer" containerID="fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078"
Nov 04 03:50:32.984264 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.984121    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:50:32.988362 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.988342    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:50:32.992177 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.992091    6751 scope.go:122] "RemoveContainer" containerID="98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61"
Nov 04 03:50:32.999448 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:32.999375    6751 scope.go:122] "RemoveContainer" containerID="f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f"
Nov 04 03:50:33.007299 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.007207    6751 scope.go:122] "RemoveContainer" containerID="a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1"
Nov 04 03:50:33.016076 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.016057    6751 scope.go:122] "RemoveContainer" containerID="dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1"
Nov 04 03:50:33.019397 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019376    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:50:33.019880 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019858    6751 cpu_manager.go:414] "containerMap: removing container" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="prom-label-proxy"
Nov 04 03:50:33.019971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019883    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="prom-label-proxy"
Nov 04 03:50:33.019971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019904    6751 cpu_manager.go:414] "containerMap: removing container" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy-web"
Nov 04 03:50:33.019971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019912    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy-web"
Nov 04 03:50:33.019971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019926    6751 cpu_manager.go:414] "containerMap: removing container" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="init-config-reloader"
Nov 04 03:50:33.019971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019936    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="init-config-reloader"
Nov 04 03:50:33.019971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019955    6751 cpu_manager.go:414] "containerMap: removing container" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy-metric"
Nov 04 03:50:33.019971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019963    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy-metric"
Nov 04 03:50:33.019971 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019974    6751 cpu_manager.go:414] "containerMap: removing container" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="config-reloader"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.019981    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="config-reloader"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020004    6751 cpu_manager.go:414] "containerMap: removing container" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020011    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020021    6751 cpu_manager.go:414] "containerMap: removing container" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="alertmanager"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020027    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="alertmanager"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020095    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020103    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="alertmanager"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020109    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="prom-label-proxy"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020115    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy-web"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020120    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="init-config-reloader"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020127    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="kube-rbac-proxy-metric"
Nov 04 03:50:33.020391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.020132    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" containerName="config-reloader"
Nov 04 03:50:33.023790 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023767    6751 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-main-tls\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.023877 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023795    6751 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.023877 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023827    6751 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-web-config\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.023877 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023837    6751 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-out\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.023877 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023847    6751 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.023877 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023859    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p84j\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-kube-api-access-4p84j\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.023877 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023872    6751 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b8f136-f4a2-4c04-a35a-e60e2788d381-metrics-client-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.024115 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023882    6751 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-cluster-tls-config\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.024115 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023890    6751 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-config-volume\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.024115 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023900    6751 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b8f136-f4a2-4c04-a35a-e60e2788d381-tls-assets\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.024115 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023932    6751 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b8f136-f4a2-4c04-a35a-e60e2788d381-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.024115 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.023947    6751 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b8f136-f4a2-4c04-a35a-e60e2788d381-alertmanager-main-db\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.025336 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.025319    6751 scope.go:122] "RemoveContainer" containerID="0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103"
Nov 04 03:50:33.033249 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.033228    6751 scope.go:122] "RemoveContainer" containerID="ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2"
Nov 04 03:50:33.033586 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:33.033560    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2\": container with ID starting with ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2 not found: ID does not exist" containerID="ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2"
Nov 04 03:50:33.033666 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.033596    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2"} err="failed to get container status \"ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2\": rpc error: code = NotFound desc = could not find container \"ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2\": container with ID starting with ff2255b3a9022fc0cd0154292320e2eaecd1aa7db3936397a047772b35de1fb2 not found: ID does not exist"
Nov 04 03:50:33.033666 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.033643    6751 scope.go:122] "RemoveContainer" containerID="fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078"
Nov 04 03:50:33.033919 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:33.033890    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078\": container with ID starting with fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078 not found: ID does not exist" containerID="fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078"
Nov 04 03:50:33.033994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.033926    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078"} err="failed to get container status \"fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078\": rpc error: code = NotFound desc = could not find container \"fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078\": container with ID starting with fcc2bd6160445f93464d93b4e93f5a3aef819c54aeb8c66fbad8b684f04cb078 not found: ID does not exist"
Nov 04 03:50:33.033994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.033945    6751 scope.go:122] "RemoveContainer" containerID="98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61"
Nov 04 03:50:33.034225 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:33.034207    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61\": container with ID starting with 98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61 not found: ID does not exist" containerID="98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61"
Nov 04 03:50:33.034287 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.034229    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61"} err="failed to get container status \"98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61\": rpc error: code = NotFound desc = could not find container \"98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61\": container with ID starting with 98e43db1ec11890cca882e06750a0bf373f4a70b171b560dc60ead2c0ded8f61 not found: ID does not exist"
Nov 04 03:50:33.034287 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.034242    6751 scope.go:122] "RemoveContainer" containerID="f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f"
Nov 04 03:50:33.034458 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:33.034441    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f\": container with ID starting with f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f not found: ID does not exist" containerID="f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f"
Nov 04 03:50:33.034507 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.034462    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f"} err="failed to get container status \"f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f\": rpc error: code = NotFound desc = could not find container \"f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f\": container with ID starting with f63d9c8f08a1fe962dd6e0055d99964ed35aaadff41281e09232c9ac3837d80f not found: ID does not exist"
Nov 04 03:50:33.034507 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.034474    6751 scope.go:122] "RemoveContainer" containerID="a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1"
Nov 04 03:50:33.034759 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:33.034735    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1\": container with ID starting with a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1 not found: ID does not exist" containerID="a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1"
Nov 04 03:50:33.034827 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.034768    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1"} err="failed to get container status \"a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1\": rpc error: code = NotFound desc = could not find container \"a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1\": container with ID starting with a11a1ef42747086c7023acf6c3c616663b15e1e5c042b8a149e978a26473f7f1 not found: ID does not exist"
Nov 04 03:50:33.034827 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.034787    6751 scope.go:122] "RemoveContainer" containerID="dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1"
Nov 04 03:50:33.035092 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:33.035074    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1\": container with ID starting with dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1 not found: ID does not exist" containerID="dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1"
Nov 04 03:50:33.035166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.035100    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1"} err="failed to get container status \"dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1\": rpc error: code = NotFound desc = could not find container \"dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1\": container with ID starting with dd7bc4608624233794e4dc2cac8bcd828d5c1996121a4c83b0482b9c8d23d1b1 not found: ID does not exist"
Nov 04 03:50:33.035166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.035120    6751 scope.go:122] "RemoveContainer" containerID="0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103"
Nov 04 03:50:33.035434 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:33.035414    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103\": container with ID starting with 0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103 not found: ID does not exist" containerID="0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103"
Nov 04 03:50:33.035521 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.035437    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103"} err="failed to get container status \"0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103\": rpc error: code = NotFound desc = could not find container \"0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103\": container with ID starting with 0fc08d5ae58f6acfa3103382b5b6aeaed1656e89f2231b0b12b97e937806d103 not found: ID does not exist"
Nov 04 03:50:33.048725 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.047663    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:50:33.048725 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.047846    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.052718 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.052687    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 03:50:33.053413 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.053380    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 03:50:33.053830 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.053808    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 03:50:33.053994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.053965    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 03:50:33.054144 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.054131    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 03:50:33.054236 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.054162    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 03:50:33.054236 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.054161    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 03:50:33.054367 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.054272    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5cbvs\""
Nov 04 03:50:33.054367 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.054317    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 03:50:33.057064 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.057025    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 03:50:33.125153 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125093    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-web-config\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125153 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125133    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125314 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125243    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5453641c-2d52-4eff-90d2-4020db02c44c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125314 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125288    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125391 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125316    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5453641c-2d52-4eff-90d2-4020db02c44c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125445 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125390    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5453641c-2d52-4eff-90d2-4020db02c44c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125445 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125416    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5453641c-2d52-4eff-90d2-4020db02c44c-config-out\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125524 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125477    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125524 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125504    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5453641c-2d52-4eff-90d2-4020db02c44c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125604 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125566    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125675 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125647    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125714 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125684    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-config-volume\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.125764 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.125709    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bd9l\" (UniqueName: \"kubernetes.io/projected/5453641c-2d52-4eff-90d2-4020db02c44c-kube-api-access-8bd9l\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.178112 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.178084    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:33.226263 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226236    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-oauth-serving-cert\") pod \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") "
Nov 04 03:50:33.226380 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226284    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-serving-cert\") pod \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") "
Nov 04 03:50:33.226380 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226306    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-service-ca\") pod \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") "
Nov 04 03:50:33.226465 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226436    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-kube-api-access-h5f4f\" (UniqueName: \"kubernetes.io/projected/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-kube-api-access-h5f4f\") pod \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") "
Nov 04 03:50:33.226530 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226511    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-oauth-config\") pod \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") "
Nov 04 03:50:33.226582 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226566    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-tmp\") pod \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") "
Nov 04 03:50:33.226652 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226640    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-config\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-config\") pod \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\" (UID: \"1c907e96-5900-4ede-b7a8-dbf6e6a092d0\") "
Nov 04 03:50:33.226726 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226702    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-oauth-serving-cert" pod "1c907e96-5900-4ede-b7a8-dbf6e6a092d0" (UID: "1c907e96-5900-4ede-b7a8-dbf6e6a092d0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:33.226777 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226734    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-service-ca" pod "1c907e96-5900-4ede-b7a8-dbf6e6a092d0" (UID: "1c907e96-5900-4ede-b7a8-dbf6e6a092d0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:33.226777 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226766    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.226875 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226830    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.226925 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226873    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-config-volume\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.226925 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226901    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bd9l\" (UniqueName: \"kubernetes.io/projected/5453641c-2d52-4eff-90d2-4020db02c44c-kube-api-access-8bd9l\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.227055 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226936    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-web-config\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.227055 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.226962    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.227055 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227013    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5453641c-2d52-4eff-90d2-4020db02c44c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.227055 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227042    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.227254 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227070    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5453641c-2d52-4eff-90d2-4020db02c44c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.227254 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227103    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5453641c-2d52-4eff-90d2-4020db02c44c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.227254 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227130    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5453641c-2d52-4eff-90d2-4020db02c44c-config-out\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.227254 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227155    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.227254 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227178    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5453641c-2d52-4eff-90d2-4020db02c44c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.227254 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227240    6751 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-oauth-serving-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.227548 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227259    6751 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-service-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.227606 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227589    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5453641c-2d52-4eff-90d2-4020db02c44c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.228265 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.228239    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5453641c-2d52-4eff-90d2-4020db02c44c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.228639 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.227192    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-config" pod "1c907e96-5900-4ede-b7a8-dbf6e6a092d0" (UID: "1c907e96-5900-4ede-b7a8-dbf6e6a092d0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:50:33.228732 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.228656    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-tmp" pod "1c907e96-5900-4ede-b7a8-dbf6e6a092d0" (UID: "1c907e96-5900-4ede-b7a8-dbf6e6a092d0"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:50:33.229704 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.229528    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5453641c-2d52-4eff-90d2-4020db02c44c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.230076 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.230013    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-serving-cert" pod "1c907e96-5900-4ede-b7a8-dbf6e6a092d0" (UID: "1c907e96-5900-4ede-b7a8-dbf6e6a092d0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:33.231855 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.231830    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.231955 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.231830    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-config-volume\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.231955 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.231940    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5453641c-2d52-4eff-90d2-4020db02c44c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.232068 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.231996    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.232285 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.232263    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5453641c-2d52-4eff-90d2-4020db02c44c-config-out\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.232695 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.232552    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.232695 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.232640    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.232695 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.232656    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-kube-api-access-h5f4f" pod "1c907e96-5900-4ede-b7a8-dbf6e6a092d0" (UID: "1c907e96-5900-4ede-b7a8-dbf6e6a092d0"). InnerVolumeSpecName "kube-api-access-h5f4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:50:33.232986 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.232960    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-oauth-config" pod "1c907e96-5900-4ede-b7a8-dbf6e6a092d0" (UID: "1c907e96-5900-4ede-b7a8-dbf6e6a092d0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:50:33.233267 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.233249    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.234586 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.234539    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5453641c-2d52-4eff-90d2-4020db02c44c-web-config\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.242283 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.242258    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bd9l\" (UniqueName: \"kubernetes.io/projected/5453641c-2d52-4eff-90d2-4020db02c44c-kube-api-access-8bd9l\") pod \"alertmanager-main-0\" (UID: \"5453641c-2d52-4eff-90d2-4020db02c44c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.327984 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.327942    6751 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-oauth-config\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.327984 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.327980    6751 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-tmp\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.327984 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.327990    6751 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-config\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.328219 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.327999    6751 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-console-serving-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.328219 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.328008    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5f4f\" (UniqueName: \"kubernetes.io/projected/1c907e96-5900-4ede-b7a8-dbf6e6a092d0-kube-api-access-h5f4f\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:50:33.478588 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.478509    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 03:50:33.786879 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.784235    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1b8f136-f4a2-4c04-a35a-e60e2788d381" path="/var/lib/kubelet/pods/d1b8f136-f4a2-4c04-a35a-e60e2788d381/volumes"
Nov 04 03:50:33.787373 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.787344    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 03:50:33.789701 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:50:33.789671    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5453641c_2d52_4eff_90d2_4020db02c44c.slice/crio-880a359ae4d60e1cdcb44fb90689958611e13cbfff9df44da5509543dfb139d1 WatchSource:0}: Error finding container 880a359ae4d60e1cdcb44fb90689958611e13cbfff9df44da5509543dfb139d1: Status 404 returned error can't find the container with id 880a359ae4d60e1cdcb44fb90689958611e13cbfff9df44da5509543dfb139d1
Nov 04 03:50:33.964130 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.964095    6751 generic.go:356] "Generic (PLEG): container finished" podID="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" containerID="57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192" exitCode=0
Nov 04 03:50:33.964531 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.964159    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5d7fb6c9-6gqmq"
Nov 04 03:50:33.964531 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.964186    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5d7fb6c9-6gqmq" event={"ID":"1c907e96-5900-4ede-b7a8-dbf6e6a092d0","Type":"ContainerDied","Data":"57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192"}
Nov 04 03:50:33.964531 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.964250    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5d7fb6c9-6gqmq" event={"ID":"1c907e96-5900-4ede-b7a8-dbf6e6a092d0","Type":"ContainerDied","Data":"c44fc83f2a2cdda3fbedea17326a62e08b99b077a82879132f67aa13bc444078"}
Nov 04 03:50:33.964531 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.964302    6751 scope.go:122] "RemoveContainer" containerID="57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192"
Nov 04 03:50:33.967586 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.967562    6751 generic.go:356] "Generic (PLEG): container finished" podID="5453641c-2d52-4eff-90d2-4020db02c44c" containerID="c0bb59c58058733866749b1dc04615ea2e985dfa296f06b3853ba8965b83539e" exitCode=0
Nov 04 03:50:33.967724 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.967639    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5453641c-2d52-4eff-90d2-4020db02c44c","Type":"ContainerDied","Data":"c0bb59c58058733866749b1dc04615ea2e985dfa296f06b3853ba8965b83539e"}
Nov 04 03:50:33.967724 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.967699    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5453641c-2d52-4eff-90d2-4020db02c44c","Type":"ContainerStarted","Data":"880a359ae4d60e1cdcb44fb90689958611e13cbfff9df44da5509543dfb139d1"}
Nov 04 03:50:33.973302 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.973283    6751 scope.go:122] "RemoveContainer" containerID="57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192"
Nov 04 03:50:33.973580 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:33.973555    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192\": container with ID starting with 57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192 not found: ID does not exist" containerID="57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192"
Nov 04 03:50:33.973678 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.973594    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192"} err="failed to get container status \"57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192\": rpc error: code = NotFound desc = could not find container \"57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192\": container with ID starting with 57ab61665c51464d06837e5f20773ee7a2178a96c1f8360a9b4ecf3740ffb192 not found: ID does not exist"
Nov 04 03:50:33.983836 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.983817    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5d7fb6c9-6gqmq"]
Nov 04 03:50:33.991491 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:33.991467    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c5d7fb6c9-6gqmq"]
Nov 04 03:50:34.976989 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:34.976949    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5453641c-2d52-4eff-90d2-4020db02c44c","Type":"ContainerStarted","Data":"2b934888155ed8a19374fc3a1808c4d95a4b13707d4fff198453bc3c5b5796e6"}
Nov 04 03:50:34.977457 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:34.977073    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5453641c-2d52-4eff-90d2-4020db02c44c","Type":"ContainerStarted","Data":"5e0b2df9929a28ab8490c0297064cf9816ba8e9a24af530b96b1ced5bf6f888c"}
Nov 04 03:50:34.977457 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:34.977118    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5453641c-2d52-4eff-90d2-4020db02c44c","Type":"ContainerStarted","Data":"a712e635b1b56cb91403204412cab4f9237236e15c4938133bd382560f90cf21"}
Nov 04 03:50:34.977457 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:34.977162    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5453641c-2d52-4eff-90d2-4020db02c44c","Type":"ContainerStarted","Data":"15adfcc29b81baeacfffaf848103040af8d291c7dc3bf0809966395ca158a0ca"}
Nov 04 03:50:34.977457 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:34.977200    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5453641c-2d52-4eff-90d2-4020db02c44c","Type":"ContainerStarted","Data":"287b77d61113843eea8a27eb20c5e6294e607015a3ed8c20ff2328bdf2349fc7"}
Nov 04 03:50:34.977457 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:34.977239    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5453641c-2d52-4eff-90d2-4020db02c44c","Type":"ContainerStarted","Data":"4d0f63c542e4ef545aadb611e7dba66c54d45432d1fc20cb9d3a1dcee9f962b8"}
Nov 04 03:50:35.011118 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:35.011075    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.998207046 podStartE2EDuration="3.01106425s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.857204ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:50:35.010318961 +0000 UTC m=+29.344256837" watchObservedRunningTime="2026-11-04 03:50:35.01106425 +0000 UTC m=+29.345002162"
Nov 04 03:50:35.784183 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:35.784148    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" path="/var/lib/kubelet/pods/1c907e96-5900-4ede-b7a8-dbf6e6a092d0/volumes"
Nov 04 03:50:36.989548 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:36.989505    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-mv47f" event={"ID":"83f09eac-bda3-430a-958a-2a9bef3ad8e0","Type":"ContainerStarted","Data":"dfec27b95d2d95b4535ab659487edac99fba5ec9b18f178fbc6a6114bef708fa"}
Nov 04 03:50:36.990081 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:36.989888    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:50:37.005510 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:37.005486    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-mv47f"
Nov 04 03:50:38.579565 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:38.579527    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:38.580111 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:38.579593    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:38.584823 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:38.584796    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:39.003445 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:39.003409    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-795f6cf888-8798b"
Nov 04 03:50:39.055142 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:39.055110    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-579797c45f-dt5hr"]
Nov 04 03:50:39.482085 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:39.482032    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:39.689968 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:39.689936    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:41.251359 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:41.251329    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g97j2"
Nov 04 03:50:43.850697 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:43.850603    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:44.056979 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:50:44.056942    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:50:51.085832 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:50:51.085794    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:51:04.080165 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.080111    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-579797c45f-dt5hr" podUID="99a5bcad-5137-4a71-8130-18ab8f851b71" containerName="console" containerID="cri-o://e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271" gracePeriod=15
Nov 04 03:51:04.348992 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.348970    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:51:04.474786 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.474761    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-service-ca\") pod \"99a5bcad-5137-4a71-8130-18ab8f851b71\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") "
Nov 04 03:51:04.474919 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.474804    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-console-config\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-console-config\") pod \"99a5bcad-5137-4a71-8130-18ab8f851b71\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") "
Nov 04 03:51:04.474919 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.474825    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-serving-cert\") pod \"99a5bcad-5137-4a71-8130-18ab8f851b71\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") "
Nov 04 03:51:04.474992 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.474951    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99a5bcad-5137-4a71-8130-18ab8f851b71-kube-api-access-58j2j\" (UniqueName: \"kubernetes.io/projected/99a5bcad-5137-4a71-8130-18ab8f851b71-kube-api-access-58j2j\") pod \"99a5bcad-5137-4a71-8130-18ab8f851b71\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") "
Nov 04 03:51:04.475034 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475015    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-oauth-config\") pod \"99a5bcad-5137-4a71-8130-18ab8f851b71\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") "
Nov 04 03:51:04.475161 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475140    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99a5bcad-5137-4a71-8130-18ab8f851b71-tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a5bcad-5137-4a71-8130-18ab8f851b71-tmp\") pod \"99a5bcad-5137-4a71-8130-18ab8f851b71\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") "
Nov 04 03:51:04.475243 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475159    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-console-config" pod "99a5bcad-5137-4a71-8130-18ab8f851b71" (UID: "99a5bcad-5137-4a71-8130-18ab8f851b71"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:51:04.475243 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475167    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-service-ca" pod "99a5bcad-5137-4a71-8130-18ab8f851b71" (UID: "99a5bcad-5137-4a71-8130-18ab8f851b71"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:51:04.475243 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475231    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-oauth-serving-cert\") pod \"99a5bcad-5137-4a71-8130-18ab8f851b71\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") "
Nov 04 03:51:04.475389 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475271    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-trusted-ca-bundle\") pod \"99a5bcad-5137-4a71-8130-18ab8f851b71\" (UID: \"99a5bcad-5137-4a71-8130-18ab8f851b71\") "
Nov 04 03:51:04.475389 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475368    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a5bcad-5137-4a71-8130-18ab8f851b71-tmp" pod "99a5bcad-5137-4a71-8130-18ab8f851b71" (UID: "99a5bcad-5137-4a71-8130-18ab8f851b71"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:51:04.475566 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475519    6751 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a5bcad-5137-4a71-8130-18ab8f851b71-tmp\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:51:04.475701 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475571    6751 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-service-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:51:04.475701 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475590    6751 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-console-config\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:51:04.475701 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475690    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-oauth-serving-cert" pod "99a5bcad-5137-4a71-8130-18ab8f851b71" (UID: "99a5bcad-5137-4a71-8130-18ab8f851b71"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:51:04.475875 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.475741    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-trusted-ca-bundle" pod "99a5bcad-5137-4a71-8130-18ab8f851b71" (UID: "99a5bcad-5137-4a71-8130-18ab8f851b71"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:51:04.477268 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.477247    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a5bcad-5137-4a71-8130-18ab8f851b71-kube-api-access-58j2j" pod "99a5bcad-5137-4a71-8130-18ab8f851b71" (UID: "99a5bcad-5137-4a71-8130-18ab8f851b71"). InnerVolumeSpecName "kube-api-access-58j2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:51:04.477438 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.477424    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-serving-cert" pod "99a5bcad-5137-4a71-8130-18ab8f851b71" (UID: "99a5bcad-5137-4a71-8130-18ab8f851b71"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:51:04.477496 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.477480    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-oauth-config" pod "99a5bcad-5137-4a71-8130-18ab8f851b71" (UID: "99a5bcad-5137-4a71-8130-18ab8f851b71"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:51:04.576067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.576042    6751 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-oauth-serving-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:51:04.576067 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.576066    6751 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a5bcad-5137-4a71-8130-18ab8f851b71-trusted-ca-bundle\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:51:04.576201 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.576082    6751 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-serving-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:51:04.576201 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.576094    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58j2j\" (UniqueName: \"kubernetes.io/projected/99a5bcad-5137-4a71-8130-18ab8f851b71-kube-api-access-58j2j\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:51:04.576201 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:04.576107    6751 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a5bcad-5137-4a71-8130-18ab8f851b71-console-oauth-config\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:51:05.091691 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:05.091657    6751 generic.go:356] "Generic (PLEG): container finished" podID="99a5bcad-5137-4a71-8130-18ab8f851b71" containerID="e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271" exitCode=0
Nov 04 03:51:05.092060 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:05.091712    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579797c45f-dt5hr"
Nov 04 03:51:05.092060 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:05.091740    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579797c45f-dt5hr" event={"ID":"99a5bcad-5137-4a71-8130-18ab8f851b71","Type":"ContainerDied","Data":"e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271"}
Nov 04 03:51:05.092060 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:05.091788    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579797c45f-dt5hr" event={"ID":"99a5bcad-5137-4a71-8130-18ab8f851b71","Type":"ContainerDied","Data":"bffd53b9b2b4de941a8bd7a30e50852b1b76b07f7e444242dd71fa4ae587ee90"}
Nov 04 03:51:05.092060 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:05.091807    6751 scope.go:122] "RemoveContainer" containerID="e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271"
Nov 04 03:51:05.100105 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:05.100090    6751 scope.go:122] "RemoveContainer" containerID="e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271"
Nov 04 03:51:05.100346 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:51:05.100327    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271\": container with ID starting with e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271 not found: ID does not exist" containerID="e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271"
Nov 04 03:51:05.100393 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:05.100355    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271"} err="failed to get container status \"e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271\": rpc error: code = NotFound desc = could not find container \"e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271\": container with ID starting with e98b17232ae4cf08885a50f8bb86516917a821b1d9de8ad5e0c4664230a42271 not found: ID does not exist"
Nov 04 03:51:05.112357 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:05.112335    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-579797c45f-dt5hr"]
Nov 04 03:51:05.116634 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:05.116603    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-579797c45f-dt5hr"]
Nov 04 03:51:05.782414 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:05.782388    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99a5bcad-5137-4a71-8130-18ab8f851b71" path="/var/lib/kubelet/pods/99a5bcad-5137-4a71-8130-18ab8f851b71/volumes"
Nov 04 03:51:06.234201 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:51:06.234169    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:51:21.085445 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:51:21.085408    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:51:51.086362 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:51:51.086282    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:52:06.284836 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:06.284802    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:21.087232 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:52:21.087166    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:52:26.992142 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:26.992111    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:49.673662 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:49.673607    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:49.881413 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:49.881357    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:50.428520 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.428482    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"]
Nov 04 03:52:50.428800 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.428785    6751 cpu_manager.go:414] "containerMap: removing container" podUID="99a5bcad-5137-4a71-8130-18ab8f851b71" containerName="console"
Nov 04 03:52:50.428800 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.428799    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99a5bcad-5137-4a71-8130-18ab8f851b71" containerName="console"
Nov 04 03:52:50.428957 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.428809    6751 cpu_manager.go:414] "containerMap: removing container" podUID="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" containerName="console"
Nov 04 03:52:50.428957 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.428815    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" containerName="console"
Nov 04 03:52:50.428957 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.428878    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="99a5bcad-5137-4a71-8130-18ab8f851b71" containerName="console"
Nov 04 03:52:50.428957 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.428885    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c907e96-5900-4ede-b7a8-dbf6e6a092d0" containerName="console"
Nov 04 03:52:50.430699 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.430682    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:50.435095 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.435066    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 04 03:52:50.436424 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.436377    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 04 03:52:50.436638 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.436596    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-jvtx7\""
Nov 04 03:52:50.443323 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.443302    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"]
Nov 04 03:52:50.522762 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.522738    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:50.522884 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.522776    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:50.522928 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.522874    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2bxf\" (UniqueName: \"kubernetes.io/projected/15c8e1d8-09f6-453a-af56-dd7994334f8e-kube-api-access-j2bxf\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:50.625023 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.624983    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2bxf\" (UniqueName: \"kubernetes.io/projected/15c8e1d8-09f6-453a-af56-dd7994334f8e-kube-api-access-j2bxf\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:50.625350 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.625330    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:50.625481 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.625455    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:50.625863 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.625841    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:50.625955 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.625893    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:50.640536 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.640517    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2bxf\" (UniqueName: \"kubernetes.io/projected/15c8e1d8-09f6-453a-af56-dd7994334f8e-kube-api-access-j2bxf\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:50.746735 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:50.746684    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:52:51.016099 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.016033    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"]
Nov 04 03:52:51.019830 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:52:51.019803    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c8e1d8_09f6_453a_af56_dd7994334f8e.slice/crio-4e511d8c11601309530de2f4dcc0aad74878902f855acb00c3f21b0e507004e6 WatchSource:0}: Error finding container 4e511d8c11601309530de2f4dcc0aad74878902f855acb00c3f21b0e507004e6: Status 404 returned error can't find the container with id 4e511d8c11601309530de2f4dcc0aad74878902f855acb00c3f21b0e507004e6
Nov 04 03:52:51.079760 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.079721    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"]
Nov 04 03:52:51.082630 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.082596    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.086788 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.086769    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 03:52:51.086926 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.086796    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 04 03:52:51.086926 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.086812    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 04 03:52:51.086926 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.086811    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 04 03:52:51.086926 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.086825    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 03:52:51.086926 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.086838    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 03:52:51.086926 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:52:51.086839    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:52:51.087140 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.086931    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qvwgz\""
Nov 04 03:52:51.096027 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.095999    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"]
Nov 04 03:52:51.129697 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.129676    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/805c5e84-e093-43c3-8242-62134e0d7360-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.129791 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.129716    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.129791 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.129758    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.129897 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.129880    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.129936 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.129915    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.129978 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.129936    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8k4j\" (UniqueName: \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-kube-api-access-x8k4j\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.130013 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.129990    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.195729 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.195706    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:51.231101 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.231077    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.231194 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.231119    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.231194 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.231145    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.231194 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.231168    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.231356 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.231195    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8k4j\" (UniqueName: \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-kube-api-access-x8k4j\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.231356 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.231213    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.231356 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.231232    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/805c5e84-e093-43c3-8242-62134e0d7360-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.231830 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.231804    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.233531 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.233502    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/805c5e84-e093-43c3-8242-62134e0d7360-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.233647 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.233535    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.233647 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.233557    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.233647 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.233608    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.243290 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.243271    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8k4j\" (UniqueName: \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-kube-api-access-x8k4j\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.243358 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.243304    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-lwn5j\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.392918 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.392898    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:51.397783 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.397744    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:51.450081 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.450050    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl" event={"ID":"15c8e1d8-09f6-453a-af56-dd7994334f8e","Type":"ContainerStarted","Data":"4e511d8c11601309530de2f4dcc0aad74878902f855acb00c3f21b0e507004e6"}
Nov 04 03:52:51.666674 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:51.666596    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"]
Nov 04 03:52:51.669103 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:52:51.669079    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805c5e84_e093_43c3_8242_62134e0d7360.slice/crio-17ce0258242b07a80f20e6ba81927f8a07d64b0a921cf6866f87c43622ee3992 WatchSource:0}: Error finding container 17ce0258242b07a80f20e6ba81927f8a07d64b0a921cf6866f87c43622ee3992: Status 404 returned error can't find the container with id 17ce0258242b07a80f20e6ba81927f8a07d64b0a921cf6866f87c43622ee3992
Nov 04 03:52:52.458313 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:52.458277    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j" event={"ID":"805c5e84-e093-43c3-8242-62134e0d7360","Type":"ContainerStarted","Data":"17ce0258242b07a80f20e6ba81927f8a07d64b0a921cf6866f87c43622ee3992"}
Nov 04 03:52:52.584815 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:52.584783    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:55.094535 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:55.094501    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:52:55.094865 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:55.094565    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:52:55.477512 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:55.477465    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j" event={"ID":"805c5e84-e093-43c3-8242-62134e0d7360","Type":"ContainerStarted","Data":"412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8"}
Nov 04 03:52:55.477726 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:55.477594    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:55.659896 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:55.659860    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:55.892110 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:55.892076    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:56.483168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:56.483122    6751 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-lwn5j container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:52:56.483614 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:56.483195    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j" podUID="805c5e84-e093-43c3-8242-62134e0d7360" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:52:56.842196 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:56.842153    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:57.050122 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:57.050021    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:57.272454 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:57.272423    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:52:58.488801 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:58.488769    6751 generic.go:356] "Generic (PLEG): container finished" podID="15c8e1d8-09f6-453a-af56-dd7994334f8e" containerID="76429f020ac4c98869019588c26694855ae8bfc8b2097332c47ede502bd7bc3a" exitCode=0
Nov 04 03:52:58.489169 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:58.488851    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl" event={"ID":"15c8e1d8-09f6-453a-af56-dd7994334f8e","Type":"ContainerDied","Data":"76429f020ac4c98869019588c26694855ae8bfc8b2097332c47ede502bd7bc3a"}
Nov 04 03:52:58.509899 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:58.509861    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j" podStartSLOduration=4.087343609 podStartE2EDuration="7.509846564s" totalImagesPullingTime="3.422502955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:52:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:52:55.513183296 +0000 UTC m=+169.847121173" watchObservedRunningTime="2026-11-04 03:52:58.509846564 +0000 UTC m=+172.843784436"
Nov 04 03:52:59.482933 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:59.482907    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:52:59.497806 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:59.497782    6751 generic.go:356] "Generic (PLEG): container finished" podID="15c8e1d8-09f6-453a-af56-dd7994334f8e" containerID="93acfe784e75f525aae035d053af1f890208afb642be50632d0057620a887bfb" exitCode=0
Nov 04 03:52:59.498150 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:59.497854    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl" event={"ID":"15c8e1d8-09f6-453a-af56-dd7994334f8e","Type":"ContainerDied","Data":"93acfe784e75f525aae035d053af1f890208afb642be50632d0057620a887bfb"}
Nov 04 03:52:59.581865 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:52:59.581812    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:06.346535 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:06.346498    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:07.531361 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:07.531325    6751 generic.go:356] "Generic (PLEG): container finished" podID="15c8e1d8-09f6-453a-af56-dd7994334f8e" containerID="432e0c7a6bdb125c434f74aaefc5cbf2a86700286e1e343e2c719157f4dc6476" exitCode=0
Nov 04 03:53:07.531769 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:07.531395    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl" event={"ID":"15c8e1d8-09f6-453a-af56-dd7994334f8e","Type":"ContainerDied","Data":"432e0c7a6bdb125c434f74aaefc5cbf2a86700286e1e343e2c719157f4dc6476"}
Nov 04 03:53:08.658236 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:08.658212    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:53:08.777999 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:08.777971    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15c8e1d8-09f6-453a-af56-dd7994334f8e-kube-api-access-j2bxf\" (UniqueName: \"kubernetes.io/projected/15c8e1d8-09f6-453a-af56-dd7994334f8e-kube-api-access-j2bxf\") pod \"15c8e1d8-09f6-453a-af56-dd7994334f8e\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") "
Nov 04 03:53:08.778132 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:08.778019    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-util\" (UniqueName: \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-util\") pod \"15c8e1d8-09f6-453a-af56-dd7994334f8e\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") "
Nov 04 03:53:08.778132 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:08.778065    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-bundle\" (UniqueName: \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-bundle\") pod \"15c8e1d8-09f6-453a-af56-dd7994334f8e\" (UID: \"15c8e1d8-09f6-453a-af56-dd7994334f8e\") "
Nov 04 03:53:08.778591 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:08.778565    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-bundle" pod "15c8e1d8-09f6-453a-af56-dd7994334f8e" (UID: "15c8e1d8-09f6-453a-af56-dd7994334f8e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:53:08.780022 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:08.780002    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c8e1d8-09f6-453a-af56-dd7994334f8e-kube-api-access-j2bxf" pod "15c8e1d8-09f6-453a-af56-dd7994334f8e" (UID: "15c8e1d8-09f6-453a-af56-dd7994334f8e"). InnerVolumeSpecName "kube-api-access-j2bxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:53:08.782141 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:08.782111    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-util" pod "15c8e1d8-09f6-453a-af56-dd7994334f8e" (UID: "15c8e1d8-09f6-453a-af56-dd7994334f8e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:53:08.878857 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:08.878835    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2bxf\" (UniqueName: \"kubernetes.io/projected/15c8e1d8-09f6-453a-af56-dd7994334f8e-kube-api-access-j2bxf\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:53:08.878857 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:08.878855    6751 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-util\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:53:08.878975 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:08.878865    6751 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15c8e1d8-09f6-453a-af56-dd7994334f8e-bundle\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:53:09.540175 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:09.540146    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl"
Nov 04 03:53:09.540316 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:09.540146    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hs4nl" event={"ID":"15c8e1d8-09f6-453a-af56-dd7994334f8e","Type":"ContainerDied","Data":"4e511d8c11601309530de2f4dcc0aad74878902f855acb00c3f21b0e507004e6"}
Nov 04 03:53:09.540316 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:09.540262    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e511d8c11601309530de2f4dcc0aad74878902f855acb00c3f21b0e507004e6"
Nov 04 03:53:21.086011 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:53:21.085932    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:53:28.636227 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:28.636190    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:30.292414 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:30.292381    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:30.499813 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:30.499784    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:32.238531 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:32.238495    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:34.463422 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:34.463378    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:34.673782 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:34.673744    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:35.439590 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.439550    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:35.700549 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.700462    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:35.739552 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.739524    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"]
Nov 04 03:53:35.739925 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.739907    6751 cpu_manager.go:414] "containerMap: removing container" podUID="15c8e1d8-09f6-453a-af56-dd7994334f8e" containerName="extract"
Nov 04 03:53:35.740013 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.739926    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15c8e1d8-09f6-453a-af56-dd7994334f8e" containerName="extract"
Nov 04 03:53:35.740013 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.739945    6751 cpu_manager.go:414] "containerMap: removing container" podUID="15c8e1d8-09f6-453a-af56-dd7994334f8e" containerName="util"
Nov 04 03:53:35.740013 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.739952    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15c8e1d8-09f6-453a-af56-dd7994334f8e" containerName="util"
Nov 04 03:53:35.740013 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.739969    6751 cpu_manager.go:414] "containerMap: removing container" podUID="15c8e1d8-09f6-453a-af56-dd7994334f8e" containerName="pull"
Nov 04 03:53:35.740013 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.739977    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15c8e1d8-09f6-453a-af56-dd7994334f8e" containerName="pull"
Nov 04 03:53:35.740443 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.740051    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="15c8e1d8-09f6-453a-af56-dd7994334f8e" containerName="extract"
Nov 04 03:53:35.743260 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.743243    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.763066 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.763046    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 03:53:35.763181 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.763097    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 03:53:35.764448 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.764431    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 03:53:35.764549 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.764481    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 03:53:35.764596 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.764577    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 03:53:35.764689 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.764676    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-9csf6\""
Nov 04 03:53:35.801888 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.801856    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"]
Nov 04 03:53:35.875696 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.875662    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49f01ef9-ae7f-4fbc-85ed-72c857421e68-manifests\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.875696 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.875697    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/49f01ef9-ae7f-4fbc-85ed-72c857421e68-metrics-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.875910 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.875731    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49f01ef9-ae7f-4fbc-85ed-72c857421e68-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.875910 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.875858    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gljfq\" (UniqueName: \"kubernetes.io/projected/49f01ef9-ae7f-4fbc-85ed-72c857421e68-kube-api-access-gljfq\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.875910 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.875880    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49f01ef9-ae7f-4fbc-85ed-72c857421e68-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.876073 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.875935    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49f01ef9-ae7f-4fbc-85ed-72c857421e68-tmp\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.976574 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.976496    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49f01ef9-ae7f-4fbc-85ed-72c857421e68-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.976741 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.976587    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gljfq\" (UniqueName: \"kubernetes.io/projected/49f01ef9-ae7f-4fbc-85ed-72c857421e68-kube-api-access-gljfq\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.976741 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.976639    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49f01ef9-ae7f-4fbc-85ed-72c857421e68-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.976741 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.976672    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49f01ef9-ae7f-4fbc-85ed-72c857421e68-tmp\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.976741 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.976713    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49f01ef9-ae7f-4fbc-85ed-72c857421e68-manifests\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.976741 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.976738    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/49f01ef9-ae7f-4fbc-85ed-72c857421e68-metrics-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.977145 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.977112    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49f01ef9-ae7f-4fbc-85ed-72c857421e68-manifests\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.979080 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.979057    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49f01ef9-ae7f-4fbc-85ed-72c857421e68-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.979205 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.979088    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49f01ef9-ae7f-4fbc-85ed-72c857421e68-tmp\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.979269 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.979233    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49f01ef9-ae7f-4fbc-85ed-72c857421e68-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.979269 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.979254    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/49f01ef9-ae7f-4fbc-85ed-72c857421e68-metrics-cert\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:35.989298 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:35.989275    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gljfq\" (UniqueName: \"kubernetes.io/projected/49f01ef9-ae7f-4fbc-85ed-72c857421e68-kube-api-access-gljfq\") pod \"opendatahub-operator-controller-manager-74f9c8978b-mqnj9\" (UID: \"49f01ef9-ae7f-4fbc-85ed-72c857421e68\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:36.022502 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:36.022469    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:36.057241 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:36.057208    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:36.232469 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:36.232370    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:36.366360 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:36.366329    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"]
Nov 04 03:53:36.369838 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:53:36.369803    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f01ef9_ae7f_4fbc_85ed_72c857421e68.slice/crio-3754417eec5f8bcfc655a462e77be487ea8084f2dbde4035346bd5dc8f985340 WatchSource:0}: Error finding container 3754417eec5f8bcfc655a462e77be487ea8084f2dbde4035346bd5dc8f985340: Status 404 returned error can't find the container with id 3754417eec5f8bcfc655a462e77be487ea8084f2dbde4035346bd5dc8f985340
Nov 04 03:53:36.439235 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:36.439204    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:36.635804 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:36.635764    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9" event={"ID":"49f01ef9-ae7f-4fbc-85ed-72c857421e68","Type":"ContainerStarted","Data":"3754417eec5f8bcfc655a462e77be487ea8084f2dbde4035346bd5dc8f985340"}
Nov 04 03:53:36.652799 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:36.652769    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:36.879880 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:36.879845    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:37.140752 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:37.140655    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:41.666947 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:41.666861    6751 generic.go:356] "Generic (PLEG): container finished" podID="49f01ef9-ae7f-4fbc-85ed-72c857421e68" containerID="e655562ebcbe1b81c54221d24ffa140e4fd569756655ecaa6a144a62d045c22c" exitCode=0
Nov 04 03:53:41.667365 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:41.666948    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9" event={"ID":"49f01ef9-ae7f-4fbc-85ed-72c857421e68","Type":"ContainerDied","Data":"e655562ebcbe1b81c54221d24ffa140e4fd569756655ecaa6a144a62d045c22c"}
Nov 04 03:53:42.078580 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:42.078523    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:42.672128 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:42.672092    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9" event={"ID":"49f01ef9-ae7f-4fbc-85ed-72c857421e68","Type":"ContainerStarted","Data":"7859fde2a8d0dc1f85dce411f7ae2fecd902feee147dea3e01a05041b18da8a2"}
Nov 04 03:53:42.672522 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:42.672212    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:42.717701 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:42.717642    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9" podStartSLOduration=2.38166123 podStartE2EDuration="7.717604293s" totalImagesPullingTime="4.916445292s" totalInitContainerRuntime="419.497771ms" isStatefulPod=false podCreationTimestamp="2026-11-04 03:53:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:53:42.715328925 +0000 UTC m=+217.049266802" watchObservedRunningTime="2026-11-04 03:53:42.717604293 +0000 UTC m=+217.051542225"
Nov 04 03:53:43.028770 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:43.028678    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:43.371063 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:43.371032    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:43.586768 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:43.586737    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:43.808548 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:43.808465    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:44.020193 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:44.020165    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:44.275908 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:44.275866    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:46.091841 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:46.091807    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:46.297420 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:46.297379    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:47.307949 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:47.307915    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:48.497257 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:48.497225    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:51.085973 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:53:51.085939    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:53:53.679634 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:53.679594    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-74f9c8978b-mqnj9"
Nov 04 03:53:53.731139 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:53.731111    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:53.937630 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:53.937548    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:54.146283 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:54.146246    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:54.830896 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:54.830855    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:55.035964 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:55.035934    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:56.573720 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:56.573677    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:56.783326 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:56.783297    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:57.859868 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:57.859835    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:58.066350 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:58.066321    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:58.955179 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:58.955150    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:53:59.163124 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:53:59.163094    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:02.449893 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:02.449856    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:02.666480 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:02.666444    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:02.887409 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:02.887379    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:03.139823 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.139754    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"]
Nov 04 03:54:03.142898 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.142878    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"
Nov 04 03:54:03.150543 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.150521    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 04 03:54:03.151104 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.151089    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xz5j4\""
Nov 04 03:54:03.151164 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.151157    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 04 03:54:03.154533 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.154509    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"]
Nov 04 03:54:03.190038 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.190013    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/494d4c6a-da7b-4c10-9f81-6087d798710e-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-g8vzm\" (UID: \"494d4c6a-da7b-4c10-9f81-6087d798710e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"
Nov 04 03:54:03.190146 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.190067    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ls2w\" (UniqueName: \"kubernetes.io/projected/494d4c6a-da7b-4c10-9f81-6087d798710e-kube-api-access-4ls2w\") pod \"cert-manager-cainjector-5c57bbd4fc-g8vzm\" (UID: \"494d4c6a-da7b-4c10-9f81-6087d798710e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"
Nov 04 03:54:03.290749 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.290720    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/494d4c6a-da7b-4c10-9f81-6087d798710e-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-g8vzm\" (UID: \"494d4c6a-da7b-4c10-9f81-6087d798710e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"
Nov 04 03:54:03.290839 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.290769    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ls2w\" (UniqueName: \"kubernetes.io/projected/494d4c6a-da7b-4c10-9f81-6087d798710e-kube-api-access-4ls2w\") pod \"cert-manager-cainjector-5c57bbd4fc-g8vzm\" (UID: \"494d4c6a-da7b-4c10-9f81-6087d798710e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"
Nov 04 03:54:03.303366 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.303342    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/494d4c6a-da7b-4c10-9f81-6087d798710e-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-g8vzm\" (UID: \"494d4c6a-da7b-4c10-9f81-6087d798710e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"
Nov 04 03:54:03.303575 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.303551    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ls2w\" (UniqueName: \"kubernetes.io/projected/494d4c6a-da7b-4c10-9f81-6087d798710e-kube-api-access-4ls2w\") pod \"cert-manager-cainjector-5c57bbd4fc-g8vzm\" (UID: \"494d4c6a-da7b-4c10-9f81-6087d798710e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"
Nov 04 03:54:03.464884 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.464812    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"
Nov 04 03:54:03.668288 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.668261    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:03.735954 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.735877    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm"]
Nov 04 03:54:03.739055 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:54:03.739023    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod494d4c6a_da7b_4c10_9f81_6087d798710e.slice/crio-3c7a54a0fd4b15a7b04c8afd6873508ac1414a22a83e80cd0ca5cc8c6bfb3b91 WatchSource:0}: Error finding container 3c7a54a0fd4b15a7b04c8afd6873508ac1414a22a83e80cd0ca5cc8c6bfb3b91: Status 404 returned error can't find the container with id 3c7a54a0fd4b15a7b04c8afd6873508ac1414a22a83e80cd0ca5cc8c6bfb3b91
Nov 04 03:54:03.757443 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.757411    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm" event={"ID":"494d4c6a-da7b-4c10-9f81-6087d798710e","Type":"ContainerStarted","Data":"3c7a54a0fd4b15a7b04c8afd6873508ac1414a22a83e80cd0ca5cc8c6bfb3b91"}
Nov 04 03:54:03.877504 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:03.877452    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:06.340633 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:06.340585    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:06.549527 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:06.549494    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:06.757762 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:06.757717    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:07.776320 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:07.776290    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm" event={"ID":"494d4c6a-da7b-4c10-9f81-6087d798710e","Type":"ContainerStarted","Data":"7f86619aa177c5b9d1e53c38d70333a31837b97a8f9da81d48bcea3f8a704b1d"}
Nov 04 03:54:07.795877 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:07.795824    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g8vzm" podStartSLOduration=0.918900284 podStartE2EDuration="4.795808055s" totalImagesPullingTime="3.876907771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:54:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:54:07.795373943 +0000 UTC m=+242.129311822" watchObservedRunningTime="2026-11-04 03:54:07.795808055 +0000 UTC m=+242.129745933"
Nov 04 03:54:08.630699 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:08.630666    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:08.838257 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:08.838231    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:10.296557 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:10.296528    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:10.502277 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:10.502242    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:11.262336 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:11.262301    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:11.476805 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:11.476775    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:11.749489 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:11.749459    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:11.927595 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:11.927563    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"]
Nov 04 03:54:11.963819 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:11.963789    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:11.988389 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:11.988361    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"]
Nov 04 03:54:11.988541 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:11.988461    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:11.992323 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:11.992298    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9wsr5\""
Nov 04 03:54:11.993792 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:11.993769    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 04 03:54:11.993898 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:11.993777    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 04 03:54:12.062309 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.062222    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc13f463-fe9a-4482-9d0a-fac5a2b845bb-tmp\") pod \"kube-auth-proxy-5bd469f65f-4x7zx\" (UID: \"fc13f463-fe9a-4482-9d0a-fac5a2b845bb\") " pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:12.062309 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.062288    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc13f463-fe9a-4482-9d0a-fac5a2b845bb-tls-certs\") pod \"kube-auth-proxy-5bd469f65f-4x7zx\" (UID: \"fc13f463-fe9a-4482-9d0a-fac5a2b845bb\") " pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:12.062506 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.062360    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48t9r\" (UniqueName: \"kubernetes.io/projected/fc13f463-fe9a-4482-9d0a-fac5a2b845bb-kube-api-access-48t9r\") pod \"kube-auth-proxy-5bd469f65f-4x7zx\" (UID: \"fc13f463-fe9a-4482-9d0a-fac5a2b845bb\") " pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:12.163598 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.163562    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc13f463-fe9a-4482-9d0a-fac5a2b845bb-tls-certs\") pod \"kube-auth-proxy-5bd469f65f-4x7zx\" (UID: \"fc13f463-fe9a-4482-9d0a-fac5a2b845bb\") " pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:12.163598 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.163598    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48t9r\" (UniqueName: \"kubernetes.io/projected/fc13f463-fe9a-4482-9d0a-fac5a2b845bb-kube-api-access-48t9r\") pod \"kube-auth-proxy-5bd469f65f-4x7zx\" (UID: \"fc13f463-fe9a-4482-9d0a-fac5a2b845bb\") " pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:12.163886 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.163685    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc13f463-fe9a-4482-9d0a-fac5a2b845bb-tmp\") pod \"kube-auth-proxy-5bd469f65f-4x7zx\" (UID: \"fc13f463-fe9a-4482-9d0a-fac5a2b845bb\") " pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:12.166199 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.166165    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc13f463-fe9a-4482-9d0a-fac5a2b845bb-tmp\") pod \"kube-auth-proxy-5bd469f65f-4x7zx\" (UID: \"fc13f463-fe9a-4482-9d0a-fac5a2b845bb\") " pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:12.166422 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.166396    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc13f463-fe9a-4482-9d0a-fac5a2b845bb-tls-certs\") pod \"kube-auth-proxy-5bd469f65f-4x7zx\" (UID: \"fc13f463-fe9a-4482-9d0a-fac5a2b845bb\") " pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:12.175196 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.175168    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:12.179791 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.179762    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48t9r\" (UniqueName: \"kubernetes.io/projected/fc13f463-fe9a-4482-9d0a-fac5a2b845bb-kube-api-access-48t9r\") pod \"kube-auth-proxy-5bd469f65f-4x7zx\" (UID: \"fc13f463-fe9a-4482-9d0a-fac5a2b845bb\") " pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:12.235872 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.235836    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"]
Nov 04 03:54:12.258674 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.258646    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"]
Nov 04 03:54:12.258826 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.258711    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:12.265940 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.265915    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 03:54:12.266075 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.266006    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 03:54:12.266182 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.266167    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5d4vv\""
Nov 04 03:54:12.266238 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.266168    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 04 03:54:12.311771 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.311743    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"
Nov 04 03:54:12.340001 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.339978    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"]
Nov 04 03:54:12.365508 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.365474    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctkrr\" (UniqueName: \"kubernetes.io/projected/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-kube-api-access-ctkrr\") pod \"limitador-limitador-74dfcfb894-xm8dg\" (UID: \"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:12.365682 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.365659    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-config-file\") pod \"limitador-limitador-74dfcfb894-xm8dg\" (UID: \"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:12.383228 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.382921    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:12.466337 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.466303    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-config-file\") pod \"limitador-limitador-74dfcfb894-xm8dg\" (UID: \"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:12.466337 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.466341    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctkrr\" (UniqueName: \"kubernetes.io/projected/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-kube-api-access-ctkrr\") pod \"limitador-limitador-74dfcfb894-xm8dg\" (UID: \"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:12.467034 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.467010    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-config-file\") pod \"limitador-limitador-74dfcfb894-xm8dg\" (UID: \"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:12.478896 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.478867    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctkrr\" (UniqueName: \"kubernetes.io/projected/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-kube-api-access-ctkrr\") pod \"limitador-limitador-74dfcfb894-xm8dg\" (UID: \"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:12.582845 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.582768    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:12.585142 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.585111    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:12.603611 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.603580    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx"]
Nov 04 03:54:12.606344 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:54:12.606317    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc13f463_fe9a_4482_9d0a_fac5a2b845bb.slice/crio-a4aea5f6bee02571e982b92f78923361717ef98f7e81da03211144a35ddcae3c WatchSource:0}: Error finding container a4aea5f6bee02571e982b92f78923361717ef98f7e81da03211144a35ddcae3c: Status 404 returned error can't find the container with id a4aea5f6bee02571e982b92f78923361717ef98f7e81da03211144a35ddcae3c
Nov 04 03:54:12.796566 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.796528    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx" event={"ID":"fc13f463-fe9a-4482-9d0a-fac5a2b845bb","Type":"ContainerStarted","Data":"a4aea5f6bee02571e982b92f78923361717ef98f7e81da03211144a35ddcae3c"}
Nov 04 03:54:12.857348 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:12.857320    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"]
Nov 04 03:54:12.860528 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:54:12.860499    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97f5fcdd_a05c_4df1_9c70_27a1b1e4091b.slice/crio-9958ddecadfc869195efb639b6d7a16a4ecaae43d394e435b947ea91fa82f942 WatchSource:0}: Error finding container 9958ddecadfc869195efb639b6d7a16a4ecaae43d394e435b947ea91fa82f942: Status 404 returned error can't find the container with id 9958ddecadfc869195efb639b6d7a16a4ecaae43d394e435b947ea91fa82f942
Nov 04 03:54:13.380755 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:13.380713    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:13.608127 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:13.608088    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:13.803248 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:13.803159    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg" event={"ID":"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b","Type":"ContainerStarted","Data":"9958ddecadfc869195efb639b6d7a16a4ecaae43d394e435b947ea91fa82f942"}
Nov 04 03:54:13.832888 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:13.832857    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:14.048041 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:14.047956    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:14.266915 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:14.266886    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:15.816537 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:15.816467    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx" event={"ID":"fc13f463-fe9a-4482-9d0a-fac5a2b845bb","Type":"ContainerStarted","Data":"5b2f4ba7cd85db032c363c16fd867a59f4ea0481b8de7f2ac11e090257553621"}
Nov 04 03:54:15.836252 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:15.836205    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bd469f65f-4x7zx" podStartSLOduration=1.970769916 podStartE2EDuration="4.836191076s" totalImagesPullingTime="2.86542116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:54:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:54:15.834864822 +0000 UTC m=+250.168802727" watchObservedRunningTime="2026-11-04 03:54:15.836191076 +0000 UTC m=+250.170128954"
Nov 04 03:54:16.659124 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:16.659101    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:16.821577 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:16.821546    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg" event={"ID":"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b","Type":"ContainerStarted","Data":"06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858"}
Nov 04 03:54:16.822013 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:16.821836    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:16.852174 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:16.852122    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg" podStartSLOduration=1.105167694 podStartE2EDuration="4.852109769s" totalImagesPullingTime="3.746942075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:54:16.850805868 +0000 UTC m=+251.184743748" watchObservedRunningTime="2026-11-04 03:54:16.852109769 +0000 UTC m=+251.186047658"
Nov 04 03:54:16.905243 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:16.905162    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:21.086099 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:54:21.086042    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:54:21.432335 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:21.432300    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:21.645997 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:21.645963    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:27.454537 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:27.454506    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"]
Nov 04 03:54:27.455132 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:27.454693    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg" podUID="97f5fcdd-a05c-4df1-9c70-27a1b1e4091b" containerName="limitador" containerID="cri-o://06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858" gracePeriod=30
Nov 04 03:54:27.455518 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:27.455327    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:27.980196 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:27.980174    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:28.093247 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.093190    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-kube-api-access-ctkrr\" (UniqueName: \"kubernetes.io/projected/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-kube-api-access-ctkrr\") pod \"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b\" (UID: \"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b\") "
Nov 04 03:54:28.093347 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.093302    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-config-file\" (UniqueName: \"kubernetes.io/configmap/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-config-file\") pod \"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b\" (UID: \"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b\") "
Nov 04 03:54:28.093639 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.093597    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-config-file" pod "97f5fcdd-a05c-4df1-9c70-27a1b1e4091b" (UID: "97f5fcdd-a05c-4df1-9c70-27a1b1e4091b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:54:28.095351 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.095328    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-kube-api-access-ctkrr" pod "97f5fcdd-a05c-4df1-9c70-27a1b1e4091b" (UID: "97f5fcdd-a05c-4df1-9c70-27a1b1e4091b"). InnerVolumeSpecName "kube-api-access-ctkrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:54:28.194848 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.194823    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctkrr\" (UniqueName: \"kubernetes.io/projected/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-kube-api-access-ctkrr\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:54:28.194848 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.194844    6751 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b-config-file\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:54:28.867553 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.867521    6751 generic.go:356] "Generic (PLEG): container finished" podID="97f5fcdd-a05c-4df1-9c70-27a1b1e4091b" containerID="06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858" exitCode=0
Nov 04 03:54:28.867980 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.867561    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"
Nov 04 03:54:28.867980 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.867605    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg" event={"ID":"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b","Type":"ContainerDied","Data":"06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858"}
Nov 04 03:54:28.867980 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.867671    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-xm8dg" event={"ID":"97f5fcdd-a05c-4df1-9c70-27a1b1e4091b","Type":"ContainerDied","Data":"9958ddecadfc869195efb639b6d7a16a4ecaae43d394e435b947ea91fa82f942"}
Nov 04 03:54:28.867980 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.867693    6751 scope.go:122] "RemoveContainer" containerID="06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858"
Nov 04 03:54:28.875969 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.875953    6751 scope.go:122] "RemoveContainer" containerID="06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858"
Nov 04 03:54:28.876212 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:54:28.876192    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858\": container with ID starting with 06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858 not found: ID does not exist" containerID="06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858"
Nov 04 03:54:28.876261 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.876220    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858"} err="failed to get container status \"06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858\": rpc error: code = NotFound desc = could not find container \"06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858\": container with ID starting with 06af1751a9be8abe706fd8828891ad94719be2d9aa89ee76cccd74adc7a4a858 not found: ID does not exist"
Nov 04 03:54:28.888758 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.888736    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"]
Nov 04 03:54:28.894400 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:28.894376    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-xm8dg"]
Nov 04 03:54:29.783859 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:29.783832    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97f5fcdd-a05c-4df1-9c70-27a1b1e4091b" path="/var/lib/kubelet/pods/97f5fcdd-a05c-4df1-9c70-27a1b1e4091b/volumes"
Nov 04 03:54:30.273297 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:30.273270    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:30.497340 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:30.497310    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:32.942466 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:32.942431    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:33.161712 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:33.161670    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:35.111005 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.110972    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:35.363601 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.363505    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:35.379648 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.379602    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"]
Nov 04 03:54:35.379981 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.379968    6751 cpu_manager.go:414] "containerMap: removing container" podUID="97f5fcdd-a05c-4df1-9c70-27a1b1e4091b" containerName="limitador"
Nov 04 03:54:35.380027 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.379983    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97f5fcdd-a05c-4df1-9c70-27a1b1e4091b" containerName="limitador"
Nov 04 03:54:35.380061 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.380045    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="97f5fcdd-a05c-4df1-9c70-27a1b1e4091b" containerName="limitador"
Nov 04 03:54:35.384609 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.384589    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.388945 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.388903    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 04 03:54:35.390216 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.390195    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Nov 04 03:54:35.390338 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.390215    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Nov 04 03:54:35.390338 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.390218    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 04 03:54:35.390338 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.390271    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-qhv5w\""
Nov 04 03:54:35.392607 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.392576    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Nov 04 03:54:35.393188 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.393166    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"]
Nov 04 03:54:35.448554 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.448523    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969c5e32-5489-4455-8846-57e78155faf5-metrics-certs\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.448746 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.448567    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/969c5e32-5489-4455-8846-57e78155faf5-manager-config\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.448746 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.448664    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cswgm\" (UniqueName: \"kubernetes.io/projected/969c5e32-5489-4455-8846-57e78155faf5-kube-api-access-cswgm\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.448746 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.448708    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/969c5e32-5489-4455-8846-57e78155faf5-cert\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.549331 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.549292    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969c5e32-5489-4455-8846-57e78155faf5-metrics-certs\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.549537 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.549337    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/969c5e32-5489-4455-8846-57e78155faf5-manager-config\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.549537 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.549367    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cswgm\" (UniqueName: \"kubernetes.io/projected/969c5e32-5489-4455-8846-57e78155faf5-kube-api-access-cswgm\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.549537 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.549399    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/969c5e32-5489-4455-8846-57e78155faf5-cert\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.550062 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.550040    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/969c5e32-5489-4455-8846-57e78155faf5-manager-config\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.551809 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.551786    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/969c5e32-5489-4455-8846-57e78155faf5-cert\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.551889 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.551817    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969c5e32-5489-4455-8846-57e78155faf5-metrics-certs\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.562070 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.562041    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cswgm\" (UniqueName: \"kubernetes.io/projected/969c5e32-5489-4455-8846-57e78155faf5-kube-api-access-cswgm\") pod \"jobset-controller-manager-5f85d5d778-bz64z\" (UID: \"969c5e32-5489-4455-8846-57e78155faf5\") " pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.697183 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.697086    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:35.977257 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:35.977063    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"]
Nov 04 03:54:35.979883 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:54:35.979855    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969c5e32_5489_4455_8846_57e78155faf5.slice/crio-1b77e3b2826d661cc7c7c7a3cce663464d757998b8ee8989579daf70a63506a6 WatchSource:0}: Error finding container 1b77e3b2826d661cc7c7c7a3cce663464d757998b8ee8989579daf70a63506a6: Status 404 returned error can't find the container with id 1b77e3b2826d661cc7c7c7a3cce663464d757998b8ee8989579daf70a63506a6
Nov 04 03:54:36.096318 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:36.096290    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:36.300385 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:36.300291    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:36.902115 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:36.902080    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z" event={"ID":"969c5e32-5489-4455-8846-57e78155faf5","Type":"ContainerStarted","Data":"1b77e3b2826d661cc7c7c7a3cce663464d757998b8ee8989579daf70a63506a6"}
Nov 04 03:54:39.420740 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:39.420702    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:39.620186 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:39.620150    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:40.259251 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:40.259213    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:40.900493 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:40.900456    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:41.105717 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:41.105692    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:41.309094 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:41.309062    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:41.925921 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:41.925886    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z" event={"ID":"969c5e32-5489-4455-8846-57e78155faf5","Type":"ContainerStarted","Data":"fdc10ddeb5fcafe668b917f34a40e56020521acbb1985a62b5cc4614c38696a8"}
Nov 04 03:54:41.926337 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:41.926034    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:54:41.946755 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:41.946711    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z" podStartSLOduration=1.830234836 podStartE2EDuration="6.946699623s" totalImagesPullingTime="5.116464787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:54:41.945361951 +0000 UTC m=+276.279299830" watchObservedRunningTime="2026-11-04 03:54:41.946699623 +0000 UTC m=+276.280637497"
Nov 04 03:54:49.420135 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:49.420053    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:49.636597 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:49.636560    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:54:51.086676 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:54:51.086641    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:54:52.934752 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:54:52.934724    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-5f85d5d778-bz64z"
Nov 04 03:55:00.735826 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:00.735779    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:00.945168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:00.945130    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:01.152006 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:01.151971    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:05.073020 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:05.072988    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:05.286907 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:05.286871    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:06.480349 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:06.480305    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:06.707737 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:06.707701    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:10.869597 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:10.869563    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:11.078810 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.078775    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:11.222938 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.222844    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"]
Nov 04 03:55:11.230170 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.230143    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.239504 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.239478    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"]
Nov 04 03:55:11.292194 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.292166    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:11.356344 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.356313    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b030923a-85f1-432c-accd-ac50ab54f8ee-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.356493 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.356348    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b030923a-85f1-432c-accd-ac50ab54f8ee-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.356493 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.356440    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.356493 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.356490    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.356594 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.356514    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5pr6\" (UniqueName: \"kubernetes.io/projected/b030923a-85f1-432c-accd-ac50ab54f8ee-kube-api-access-q5pr6\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.356667 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.356598    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.356667 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.356636    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.456981 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.456957    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.457102 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.456997    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.457102 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.457020    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5pr6\" (UniqueName: \"kubernetes.io/projected/b030923a-85f1-432c-accd-ac50ab54f8ee-kube-api-access-q5pr6\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.457102 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.457055    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.457102 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.457071    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.457317 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.457106    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b030923a-85f1-432c-accd-ac50ab54f8ee-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.457501 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.457404    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b030923a-85f1-432c-accd-ac50ab54f8ee-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.457906 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.457878    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.459544 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.459522    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.459704 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.459688    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b030923a-85f1-432c-accd-ac50ab54f8ee-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.459755 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.459690    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.460284 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.460268    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b030923a-85f1-432c-accd-ac50ab54f8ee-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.471956 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.471931    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5pr6\" (UniqueName: \"kubernetes.io/projected/b030923a-85f1-432c-accd-ac50ab54f8ee-kube-api-access-q5pr6\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.472271 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.472254    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b030923a-85f1-432c-accd-ac50ab54f8ee-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-82hvc\" (UID: \"b030923a-85f1-432c-accd-ac50ab54f8ee\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.542896 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.542823    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:11.828995 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.828919    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"]
Nov 04 03:55:11.832548 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:55:11.832521    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb030923a_85f1_432c_accd_ac50ab54f8ee.slice/crio-94c7ed7ebb845ff6b64ae14a0aabe03cfa0d8b42fcfc9821765a91110ad8efee WatchSource:0}: Error finding container 94c7ed7ebb845ff6b64ae14a0aabe03cfa0d8b42fcfc9821765a91110ad8efee: Status 404 returned error can't find the container with id 94c7ed7ebb845ff6b64ae14a0aabe03cfa0d8b42fcfc9821765a91110ad8efee
Nov 04 03:55:11.835356 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.835322    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:55:11.835465 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:11.835392    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:55:12.043762 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:12.043725    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc" event={"ID":"b030923a-85f1-432c-accd-ac50ab54f8ee","Type":"ContainerStarted","Data":"6453081c85cb8875e6263a192958fac7bbf6c12a45e698c364a1da45b9a9e860"}
Nov 04 03:55:12.043762 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:12.043781    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc" event={"ID":"b030923a-85f1-432c-accd-ac50ab54f8ee","Type":"ContainerStarted","Data":"94c7ed7ebb845ff6b64ae14a0aabe03cfa0d8b42fcfc9821765a91110ad8efee"}
Nov 04 03:55:12.044274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:12.044012    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:12.045434 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:12.045411    6751 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-82hvc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 03:55:12.045555 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:12.045456    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc" podUID="b030923a-85f1-432c-accd-ac50ab54f8ee" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 03:55:12.073943 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:12.073892    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc" podStartSLOduration=1.073875658 podStartE2EDuration="1.073875658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:55:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:55:12.07273848 +0000 UTC m=+306.406676384" watchObservedRunningTime="2026-11-04 03:55:12.073875658 +0000 UTC m=+306.407813537"
Nov 04 03:55:12.175540 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:12.175495    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:12.387993 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:12.387951    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:12.640475 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:12.640435    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:13.048533 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.048450    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-82hvc"
Nov 04 03:55:13.115397 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.115361    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"]
Nov 04 03:55:13.115634 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.115594    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j" podUID="805c5e84-e093-43c3-8242-62134e0d7360" containerName="discovery" containerID="cri-o://412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8" gracePeriod=30
Nov 04 03:55:13.359500 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.359473    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:55:13.372327 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.372299    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-kubeconfig\") pod \"805c5e84-e093-43c3-8242-62134e0d7360\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") "
Nov 04 03:55:13.372452 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.372347    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-cacerts\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-cacerts\") pod \"805c5e84-e093-43c3-8242-62134e0d7360\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") "
Nov 04 03:55:13.372452 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.372381    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-kube-api-access-x8k4j\" (UniqueName: \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-kube-api-access-x8k4j\") pod \"805c5e84-e093-43c3-8242-62134e0d7360\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") "
Nov 04 03:55:13.372452 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.372433    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-dns-cert\") pod \"805c5e84-e093-43c3-8242-62134e0d7360\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") "
Nov 04 03:55:13.372605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.372467    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-istio-token\" (UniqueName: \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-istio-token\") pod \"805c5e84-e093-43c3-8242-62134e0d7360\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") "
Nov 04 03:55:13.372605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.372499    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-ca-configmap\") pod \"805c5e84-e093-43c3-8242-62134e0d7360\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") "
Nov 04 03:55:13.372605 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.372554    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/805c5e84-e093-43c3-8242-62134e0d7360-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/805c5e84-e093-43c3-8242-62134e0d7360-local-certs\") pod \"805c5e84-e093-43c3-8242-62134e0d7360\" (UID: \"805c5e84-e093-43c3-8242-62134e0d7360\") "
Nov 04 03:55:13.373013 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.372947    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-ca-configmap" pod "805c5e84-e093-43c3-8242-62134e0d7360" (UID: "805c5e84-e093-43c3-8242-62134e0d7360"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 03:55:13.374686 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.374649    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-dns-cert" pod "805c5e84-e093-43c3-8242-62134e0d7360" (UID: "805c5e84-e093-43c3-8242-62134e0d7360"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:55:13.374932 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.374889    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-kubeconfig" pod "805c5e84-e093-43c3-8242-62134e0d7360" (UID: "805c5e84-e093-43c3-8242-62134e0d7360"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:55:13.374932 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.374897    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-istio-token" pod "805c5e84-e093-43c3-8242-62134e0d7360" (UID: "805c5e84-e093-43c3-8242-62134e0d7360"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:55:13.375104 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.374952    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805c5e84-e093-43c3-8242-62134e0d7360-local-certs" pod "805c5e84-e093-43c3-8242-62134e0d7360" (UID: "805c5e84-e093-43c3-8242-62134e0d7360"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 03:55:13.375104 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.374993    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-cacerts" pod "805c5e84-e093-43c3-8242-62134e0d7360" (UID: "805c5e84-e093-43c3-8242-62134e0d7360"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:55:13.375104 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.374996    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-kube-api-access-x8k4j" pod "805c5e84-e093-43c3-8242-62134e0d7360" (UID: "805c5e84-e093-43c3-8242-62134e0d7360"). InnerVolumeSpecName "kube-api-access-x8k4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:55:13.473765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.473734    6751 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-ca-configmap\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:55:13.473765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.473763    6751 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/805c5e84-e093-43c3-8242-62134e0d7360-local-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:55:13.473921 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.473774    6751 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-kubeconfig\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:55:13.473921 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.473783    6751 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-cacerts\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:55:13.473921 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.473792    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8k4j\" (UniqueName: \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-kube-api-access-x8k4j\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:55:13.473921 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.473802    6751 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/805c5e84-e093-43c3-8242-62134e0d7360-istio-csr-dns-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:55:13.473921 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:13.473815    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/805c5e84-e093-43c3-8242-62134e0d7360-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:55:14.053161 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:14.053081    6751 generic.go:356] "Generic (PLEG): container finished" podID="805c5e84-e093-43c3-8242-62134e0d7360" containerID="412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8" exitCode=0
Nov 04 03:55:14.053161 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:14.053129    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"
Nov 04 03:55:14.053161 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:14.053151    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j" event={"ID":"805c5e84-e093-43c3-8242-62134e0d7360","Type":"ContainerDied","Data":"412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8"}
Nov 04 03:55:14.053819 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:14.053205    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j" event={"ID":"805c5e84-e093-43c3-8242-62134e0d7360","Type":"ContainerDied","Data":"17ce0258242b07a80f20e6ba81927f8a07d64b0a921cf6866f87c43622ee3992"}
Nov 04 03:55:14.053819 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:14.053234    6751 scope.go:122] "RemoveContainer" containerID="412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8"
Nov 04 03:55:14.061807 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:14.061791    6751 scope.go:122] "RemoveContainer" containerID="412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8"
Nov 04 03:55:14.062082 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:55:14.062062    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8\": container with ID starting with 412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8 not found: ID does not exist" containerID="412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8"
Nov 04 03:55:14.062126 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:14.062093    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8"} err="failed to get container status \"412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8\": rpc error: code = NotFound desc = could not find container \"412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8\": container with ID starting with 412b4ea1e9083008fe2a946097a1b6913137a679ffbbbd69d7c2b144e26f12a8 not found: ID does not exist"
Nov 04 03:55:14.072657 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:14.072613    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"]
Nov 04 03:55:14.076864 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:14.076841    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-lwn5j"]
Nov 04 03:55:15.787874 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:15.787838    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="805c5e84-e093-43c3-8242-62134e0d7360" path="/var/lib/kubelet/pods/805c5e84-e093-43c3-8242-62134e0d7360/volumes"
Nov 04 03:55:21.086241 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:55:21.086206    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:55:36.488102 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:36.488062    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:36.707549 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:36.707519    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:37.796731 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:37.796699    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:38.034947 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:38.034912    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:48.474812 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:48.474780    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:48.697424 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:55:48.697381    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:55:51.086850 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:55:51.086813    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:56:06.540059 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:56:06.540022    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:56:21.085892 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:56:21.085815    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:56:47.091792 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:56:47.091751    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:56:47.312611 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:56:47.312575    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:56:51.085633 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:56:51.085590    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:57:06.589927 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.589893    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:06.840550 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.840480    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"]
Nov 04 03:57:06.840854 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.840840    6751 cpu_manager.go:414] "containerMap: removing container" podUID="805c5e84-e093-43c3-8242-62134e0d7360" containerName="discovery"
Nov 04 03:57:06.840899 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.840855    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="805c5e84-e093-43c3-8242-62134e0d7360" containerName="discovery"
Nov 04 03:57:06.840938 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.840920    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="805c5e84-e093-43c3-8242-62134e0d7360" containerName="discovery"
Nov 04 03:57:06.843876 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.843860    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:06.848825 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.848782    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-68php\""
Nov 04 03:57:06.848938 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.848836    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 04 03:57:06.848938 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.848836    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 04 03:57:06.848938 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.848862    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 04 03:57:06.854148 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.854128    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"]
Nov 04 03:57:06.928418 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.928395    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:06.928546 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.928425    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9hgm\" (UniqueName: \"kubernetes.io/projected/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-kube-api-access-t9hgm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:06.928546 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.928451    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:06.928546 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.928472    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:06.928758 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.928650    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:06.928758 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:06.928688    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.000726 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.000699    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:07.029132 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.029108    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.029253 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.029139    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9hgm\" (UniqueName: \"kubernetes.io/projected/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-kube-api-access-t9hgm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.029253 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.029167    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.029253 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.029186    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.029253 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.029225    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.029253 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.029247    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.029498 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.029464    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.029551 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.029531    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.029675 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.029658    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.031516 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.031489    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.031719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.031704    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.041549 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.041528    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9hgm\" (UniqueName: \"kubernetes.io/projected/1669eab8-e98a-4eb5-838d-943fb4ba5d2b-kube-api-access-t9hgm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq\" (UID: \"1669eab8-e98a-4eb5-838d-943fb4ba5d2b\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.156075 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.156036    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:07.429150 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.429073    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"]
Nov 04 03:57:07.432123 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:57:07.432095    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1669eab8_e98a_4eb5_838d_943fb4ba5d2b.slice/crio-cc7c0e02ddb2b51088d1c5482fae8e9553c8ecacdd6e15bf02768aea029f9dff WatchSource:0}: Error finding container cc7c0e02ddb2b51088d1c5482fae8e9553c8ecacdd6e15bf02768aea029f9dff: Status 404 returned error can't find the container with id cc7c0e02ddb2b51088d1c5482fae8e9553c8ecacdd6e15bf02768aea029f9dff
Nov 04 03:57:07.433293 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.433278    6751 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 03:57:07.499711 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:07.499682    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq" event={"ID":"1669eab8-e98a-4eb5-838d-943fb4ba5d2b","Type":"ContainerStarted","Data":"cc7c0e02ddb2b51088d1c5482fae8e9553c8ecacdd6e15bf02768aea029f9dff"}
Nov 04 03:57:08.069269 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:08.069234    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:09.509968 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:09.509929    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq" event={"ID":"1669eab8-e98a-4eb5-838d-943fb4ba5d2b","Type":"ContainerStarted","Data":"0af542395209d13feae0d9f55c8d31887866fa5b0e2e04e23b7b11a15e2c0839"}
Nov 04 03:57:09.530232 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:09.530172    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq" podStartSLOduration=2.101410807 podStartE2EDuration="3.530153776s" totalImagesPullingTime="1.428742969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:09.530032967 +0000 UTC m=+423.863970846" watchObservedRunningTime="2026-11-04 03:57:09.530153776 +0000 UTC m=+423.864091655"
Nov 04 03:57:10.064267 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:10.064235    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:10.156503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:10.156478    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:11.575309 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.575277    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"]
Nov 04 03:57:11.579395 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.579373    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.590216 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.590191    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Nov 04 03:57:11.591499 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.591479    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"]
Nov 04 03:57:11.667111 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.667072    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.667278 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.667121    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21a18d39-f8f7-4cc8-a946-312e07cfce01-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.667278 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.667141    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5sj2\" (UniqueName: \"kubernetes.io/projected/21a18d39-f8f7-4cc8-a946-312e07cfce01-kube-api-access-l5sj2\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.667350 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.667281    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.667350 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.667312    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.667350 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.667345    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.744140 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.744107    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:11.768803 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.768768    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.768803 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.768802    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.769006 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.768835    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.769006 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.768912    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.769006 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.768957    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21a18d39-f8f7-4cc8-a946-312e07cfce01-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.769006 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.768982    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5sj2\" (UniqueName: \"kubernetes.io/projected/21a18d39-f8f7-4cc8-a946-312e07cfce01-kube-api-access-l5sj2\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.769242 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.769208    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.769376 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.769297    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.769376 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.769303    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.771354 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.771333    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21a18d39-f8f7-4cc8-a946-312e07cfce01-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.771548 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.771523    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21a18d39-f8f7-4cc8-a946-312e07cfce01-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.784844 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.784817    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5sj2\" (UniqueName: \"kubernetes.io/projected/21a18d39-f8f7-4cc8-a946-312e07cfce01-kube-api-access-l5sj2\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-l5tsl\" (UID: \"21a18d39-f8f7-4cc8-a946-312e07cfce01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:11.896138 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:11.896103    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:12.172576 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:12.172500    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"]
Nov 04 03:57:12.175265 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:57:12.175240    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21a18d39_f8f7_4cc8_a946_312e07cfce01.slice/crio-cac57162e4c5c301114b29b449b7db99eef88503c78608253cac9cd719f91bff WatchSource:0}: Error finding container cac57162e4c5c301114b29b449b7db99eef88503c78608253cac9cd719f91bff: Status 404 returned error can't find the container with id cac57162e4c5c301114b29b449b7db99eef88503c78608253cac9cd719f91bff
Nov 04 03:57:12.523678 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:12.523646    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl" event={"ID":"21a18d39-f8f7-4cc8-a946-312e07cfce01","Type":"ContainerStarted","Data":"a4b219542b4ec4f2cacff1b7282267ce6aa0455e413a8d389a313aba45f16462"}
Nov 04 03:57:12.523808 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:12.523706    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl" event={"ID":"21a18d39-f8f7-4cc8-a946-312e07cfce01","Type":"ContainerStarted","Data":"cac57162e4c5c301114b29b449b7db99eef88503c78608253cac9cd719f91bff"}
Nov 04 03:57:12.545332 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:12.545291    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl" podStartSLOduration=1.2612455 podStartE2EDuration="1.545276014s" totalImagesPullingTime="284.030514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:12.543677273 +0000 UTC m=+426.877615153" watchObservedRunningTime="2026-11-04 03:57:12.545276014 +0000 UTC m=+426.879213892"
Nov 04 03:57:12.896799 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:12.896759    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:15.198649 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:15.198562    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:16.143748 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.143714    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"]
Nov 04 03:57:16.147002 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.146981    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.155456 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.155425    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Nov 04 03:57:16.160103 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.160072    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"]
Nov 04 03:57:16.310542 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.310499    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecc0ca03-4870-4c00-a872-f2d0928bd23c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.311043 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.310573    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.311043 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.310664    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.311043 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.310713    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzks8\" (UniqueName: \"kubernetes.io/projected/ecc0ca03-4870-4c00-a872-f2d0928bd23c-kube-api-access-mzks8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.311043 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.310736    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.311043 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.310825    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.412259 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.412163    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.412259 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.412222    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.412483 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.412276    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecc0ca03-4870-4c00-a872-f2d0928bd23c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.412483 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.412336    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.412483 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.412365    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.412483 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.412432    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzks8\" (UniqueName: \"kubernetes.io/projected/ecc0ca03-4870-4c00-a872-f2d0928bd23c-kube-api-access-mzks8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.412719 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.412664    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.412822 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.412799    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.412882 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.412842    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.414655 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.414614    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecc0ca03-4870-4c00-a872-f2d0928bd23c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.414815 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.414796    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecc0ca03-4870-4c00-a872-f2d0928bd23c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.417834 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.417811    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:16.433071 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.433049    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzks8\" (UniqueName: \"kubernetes.io/projected/ecc0ca03-4870-4c00-a872-f2d0928bd23c-kube-api-access-mzks8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2\" (UID: \"ecc0ca03-4870-4c00-a872-f2d0928bd23c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.463098 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.463073    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:16.736707 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.736610    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"]
Nov 04 03:57:16.739580 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:57:16.739549    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc0ca03_4870_4c00_a872_f2d0928bd23c.slice/crio-ae6963b1c977a12dd735f4bd27d2dfef2f210b50ba4f7f99f266b410368aab21 WatchSource:0}: Error finding container ae6963b1c977a12dd735f4bd27d2dfef2f210b50ba4f7f99f266b410368aab21: Status 404 returned error can't find the container with id ae6963b1c977a12dd735f4bd27d2dfef2f210b50ba4f7f99f266b410368aab21
Nov 04 03:57:16.787725 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.787700    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-zn2mw"]
Nov 04 03:57:16.792317 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.792291    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:16.796679 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.796655    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Nov 04 03:57:16.801205 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.801173    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-zn2mw"]
Nov 04 03:57:16.916055 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.916025    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-home\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:16.916183 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.916088    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkncm\" (UniqueName: \"kubernetes.io/projected/2dcaadca-0d52-4448-a2ad-51788bbeaa80-kube-api-access-wkncm\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:16.916237 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.916186    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:16.916237 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.916220    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:16.916316 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.916244    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dcaadca-0d52-4448-a2ad-51788bbeaa80-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:16.916316 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.916296    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:16.941469 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:16.941440    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:17.016881 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.016804    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.016881 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.016852    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dcaadca-0d52-4448-a2ad-51788bbeaa80-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.017097 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.016882    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.017097 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.016943    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-home\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.017097 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.017036    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkncm\" (UniqueName: \"kubernetes.io/projected/2dcaadca-0d52-4448-a2ad-51788bbeaa80-kube-api-access-wkncm\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.017097 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.017088    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.017393 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.017333    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.017393 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.017374    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-home\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.017579 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.017418    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.019250 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.019221    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dcaadca-0d52-4448-a2ad-51788bbeaa80-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.019615 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.019593    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dcaadca-0d52-4448-a2ad-51788bbeaa80-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.029475 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.029453    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkncm\" (UniqueName: \"kubernetes.io/projected/2dcaadca-0d52-4448-a2ad-51788bbeaa80-kube-api-access-wkncm\") pod \"echo-simulator-kserve-64b6f8f96c-zn2mw\" (UID: \"2dcaadca-0d52-4448-a2ad-51788bbeaa80\") " pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.106055 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.106028    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:17.157333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.156747    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:17.172943 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.172917    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:17.380162 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.380130    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-zn2mw"]
Nov 04 03:57:17.383278 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:57:17.383253    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dcaadca_0d52_4448_a2ad_51788bbeaa80.slice/crio-30ee51a740ccacd498ddd106ea493a5fda30e9368a978f0b8b6d703730931a00 WatchSource:0}: Error finding container 30ee51a740ccacd498ddd106ea493a5fda30e9368a978f0b8b6d703730931a00: Status 404 returned error can't find the container with id 30ee51a740ccacd498ddd106ea493a5fda30e9368a978f0b8b6d703730931a00
Nov 04 03:57:17.545940 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.545902    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2" event={"ID":"ecc0ca03-4870-4c00-a872-f2d0928bd23c","Type":"ContainerStarted","Data":"8250ddbc3874ff1cf2100967ee51ac070283f15de637c5567e0047684c777a56"}
Nov 04 03:57:17.546134 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.545958    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2" event={"ID":"ecc0ca03-4870-4c00-a872-f2d0928bd23c","Type":"ContainerStarted","Data":"ae6963b1c977a12dd735f4bd27d2dfef2f210b50ba4f7f99f266b410368aab21"}
Nov 04 03:57:17.547179 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.547150    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw" event={"ID":"2dcaadca-0d52-4448-a2ad-51788bbeaa80","Type":"ContainerStarted","Data":"30ee51a740ccacd498ddd106ea493a5fda30e9368a978f0b8b6d703730931a00"}
Nov 04 03:57:17.558353 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.558328    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq"
Nov 04 03:57:17.567042 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:17.566998    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2" podStartSLOduration=1.286241157 podStartE2EDuration="1.566986701s" totalImagesPullingTime="280.745544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:17.56615964 +0000 UTC m=+431.900097519" watchObservedRunningTime="2026-11-04 03:57:17.566986701 +0000 UTC m=+431.900924579"
Nov 04 03:57:18.463469 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:18.463428    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:18.998965 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:18.998941    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:19.557093 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:19.557053    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw" event={"ID":"2dcaadca-0d52-4448-a2ad-51788bbeaa80","Type":"ContainerStarted","Data":"53c341dc60a0ee3e57f552fbdbd66b7a3e135b3ff2c6337cb7ab789616e52589"}
Nov 04 03:57:19.577950 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:19.577892    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw" podStartSLOduration=1.972331794 podStartE2EDuration="3.577875252s" totalImagesPullingTime="1.605543458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:19.576989618 +0000 UTC m=+433.910927497" watchObservedRunningTime="2026-11-04 03:57:19.577875252 +0000 UTC m=+433.911813131"
Nov 04 03:57:20.106697 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:20.106663    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:20.871483 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:20.871447    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:21.086747 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:57:21.086709    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:57:21.093868 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:21.093843    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:21.896930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:21.896893    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:21.909053 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:21.909029    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:22.581160 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:22.581130    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-l5tsl"
Nov 04 03:57:26.463844 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:26.463810    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:26.476248 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:26.476225    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:26.599088 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:26.599062    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2"
Nov 04 03:57:27.067034 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.066999    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"]
Nov 04 03:57:27.071047 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.071017    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"]
Nov 04 03:57:27.071185 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.071095    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.074551 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.074531    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.083994 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.083975    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"]
Nov 04 03:57:27.085529 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.085505    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"]
Nov 04 03:57:27.106234 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.106212    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:27.118901 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.118880    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:27.155389 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.155364    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mb97r\""
Nov 04 03:57:27.207786 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.207756    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.207921 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.207795    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.207921 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.207889    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.208038 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.207973    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7427f8fe-4e90-426b-9e23-7054641123b3-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.208038 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208009    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.208112 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208091    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.208156 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208119    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nhmg\" (UniqueName: \"kubernetes.io/projected/90da3418-09f9-4d87-b646-75914ed6a3bc-kube-api-access-9nhmg\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.208298 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208283    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.208374 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208359    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.208423 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208394    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90da3418-09f9-4d87-b646-75914ed6a3bc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.208461 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208445    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7s57\" (UniqueName: \"kubernetes.io/projected/7427f8fe-4e90-426b-9e23-7054641123b3-kube-api-access-g7s57\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.208525 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208510    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.208563 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208551    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.208613 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208568    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.208613 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208599    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.208738 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208709    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7427f8fe-4e90-426b-9e23-7054641123b3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.208798 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208737    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.208798 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.208753    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7427f8fe-4e90-426b-9e23-7054641123b3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.309349 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309309    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.309349 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309353    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.309594 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309377    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.309594 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309421    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7427f8fe-4e90-426b-9e23-7054641123b3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.309594 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309454    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.309594 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309475    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7427f8fe-4e90-426b-9e23-7054641123b3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.309594 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309521    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.309594 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309558    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.309930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309604    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.309930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309655    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7427f8fe-4e90-426b-9e23-7054641123b3-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.309930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309680    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.309930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309722    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.309930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309752    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhmg\" (UniqueName: \"kubernetes.io/projected/90da3418-09f9-4d87-b646-75914ed6a3bc-kube-api-access-9nhmg\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.309930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309784    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.309930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309815    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.309930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309815    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.309930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309830    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.309930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309913    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.310442 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309838    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.310442 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.309996    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90da3418-09f9-4d87-b646-75914ed6a3bc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.310442 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.310037    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.310442 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.310039    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7s57\" (UniqueName: \"kubernetes.io/projected/7427f8fe-4e90-426b-9e23-7054641123b3-kube-api-access-g7s57\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.310442 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.310066    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.310442 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.310086    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.310442 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.310126    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.310442 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.310204    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.310877 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.310850    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90da3418-09f9-4d87-b646-75914ed6a3bc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.311001 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.310937    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7427f8fe-4e90-426b-9e23-7054641123b3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.312417 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.312393    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7427f8fe-4e90-426b-9e23-7054641123b3-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.312504 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.312484    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.312683 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.312665    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7427f8fe-4e90-426b-9e23-7054641123b3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.312826 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.312803    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.324158 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.324101    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7427f8fe-4e90-426b-9e23-7054641123b3-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.324445 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.324415    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7s57\" (UniqueName: \"kubernetes.io/projected/7427f8fe-4e90-426b-9e23-7054641123b3-kube-api-access-g7s57\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-wjbj2\" (UID: \"7427f8fe-4e90-426b-9e23-7054641123b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.324593 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.324572    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nhmg\" (UniqueName: \"kubernetes.io/projected/90da3418-09f9-4d87-b646-75914ed6a3bc-kube-api-access-9nhmg\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.324774 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.324757    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90da3418-09f9-4d87-b646-75914ed6a3bc-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-xrqpp\" (UID: \"90da3418-09f9-4d87-b646-75914ed6a3bc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.386241 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.386216    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:27.396233 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.396214    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:27.604862 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.604838    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-zn2mw"
Nov 04 03:57:27.670024 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.669990    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"]
Nov 04 03:57:27.673414 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:57:27.673383    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90da3418_09f9_4d87_b646_75914ed6a3bc.slice/crio-9cd92d916da714a409c22bea201f19aeb1b96dfe353aa155ce63cc592607835d WatchSource:0}: Error finding container 9cd92d916da714a409c22bea201f19aeb1b96dfe353aa155ce63cc592607835d: Status 404 returned error can't find the container with id 9cd92d916da714a409c22bea201f19aeb1b96dfe353aa155ce63cc592607835d
Nov 04 03:57:27.693044 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:27.693017    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"]
Nov 04 03:57:27.696210 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:57:27.696179    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7427f8fe_4e90_426b_9e23_7054641123b3.slice/crio-c66d79b6bf224b4b02455993b2a28ba221a4159bd72cd1173eb7366a2bdf228e WatchSource:0}: Error finding container c66d79b6bf224b4b02455993b2a28ba221a4159bd72cd1173eb7366a2bdf228e: Status 404 returned error can't find the container with id c66d79b6bf224b4b02455993b2a28ba221a4159bd72cd1173eb7366a2bdf228e
Nov 04 03:57:28.599143 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:28.599107    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" event={"ID":"7427f8fe-4e90-426b-9e23-7054641123b3","Type":"ContainerStarted","Data":"c66d79b6bf224b4b02455993b2a28ba221a4159bd72cd1173eb7366a2bdf228e"}
Nov 04 03:57:28.600429 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:28.600389    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp" event={"ID":"90da3418-09f9-4d87-b646-75914ed6a3bc","Type":"ContainerStarted","Data":"9cd92d916da714a409c22bea201f19aeb1b96dfe353aa155ce63cc592607835d"}
Nov 04 03:57:31.286206 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:31.286170    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:57:31.286510 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:31.286242    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:57:31.286510 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:31.286278    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:57:31.380296 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:31.380260    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:57:31.380404 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:31.380382    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:57:31.380471 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:31.380431    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:57:31.620427 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:31.620389    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" event={"ID":"7427f8fe-4e90-426b-9e23-7054641123b3","Type":"ContainerStarted","Data":"cdb611d6e8ce6a27793d0ee4b76f9da59d71e1c99c34c3e095f6b31a6d3de423"}
Nov 04 03:57:31.622777 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:31.622742    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp" event={"ID":"90da3418-09f9-4d87-b646-75914ed6a3bc","Type":"ContainerStarted","Data":"90ef6c155a549a636c8fc8118609c6acefe2f9e217540cf837445e8cf295d7a7"}
Nov 04 03:57:31.646168 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:31.646107    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" podStartSLOduration=0.964680247 podStartE2EDuration="4.646083483s" totalImagesPullingTime="3.681403236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:31.64566518 +0000 UTC m=+445.979603063" watchObservedRunningTime="2026-11-04 03:57:31.646083483 +0000 UTC m=+445.980021362"
Nov 04 03:57:31.668940 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:31.668884    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp" podStartSLOduration=1.059683883 podStartE2EDuration="4.668866909s" totalImagesPullingTime="3.609183026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:31.667794538 +0000 UTC m=+446.001732420" watchObservedRunningTime="2026-11-04 03:57:31.668866909 +0000 UTC m=+446.002804788"
Nov 04 03:57:32.388646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:32.388443    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:32.388646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:32.388583    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:57:32.389221 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:32.388661    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp" podUID="90da3418-09f9-4d87-b646-75914ed6a3bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 04 03:57:32.399682 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:32.398315    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:32.399860 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:32.399792    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:57:32.399899 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:32.399856    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" podUID="7427f8fe-4e90-426b-9e23-7054641123b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 03:57:33.390644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:33.389971    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:57:33.390644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:33.390034    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp" podUID="90da3418-09f9-4d87-b646-75914ed6a3bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 04 03:57:33.399646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:33.398898    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:57:33.399646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:33.398946    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" podUID="7427f8fe-4e90-426b-9e23-7054641123b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 03:57:34.392812 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:34.392770    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:57:34.393405 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:34.393376    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp" podUID="90da3418-09f9-4d87-b646-75914ed6a3bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 04 03:57:34.397735 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:34.397708    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:57:34.397914 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:34.397884    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" podUID="7427f8fe-4e90-426b-9e23-7054641123b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 03:57:34.445242 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:34.445216    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:35.387059 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.387008    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:57:35.387271 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.387090    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp" podUID="90da3418-09f9-4d87-b646-75914ed6a3bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 04 03:57:35.397007 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.396976    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:57:35.397518 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.397489    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" podUID="7427f8fe-4e90-426b-9e23-7054641123b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 03:57:35.705737 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.690466    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-hpcwt"]
Nov 04 03:57:35.713461 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.713439    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:57:35.725284 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.724557    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6gr5\" (UniqueName: \"kubernetes.io/projected/9815ccf8-1a32-46c0-9037-666e183f09a9-kube-api-access-v6gr5\") pod \"ai-gateway-controller-69464886d7-hpcwt\" (UID: \"9815ccf8-1a32-46c0-9037-666e183f09a9\") " pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:57:35.725284 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.724716    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9815ccf8-1a32-46c0-9037-666e183f09a9-metrics-certs\") pod \"ai-gateway-controller-69464886d7-hpcwt\" (UID: \"9815ccf8-1a32-46c0-9037-666e183f09a9\") " pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:57:35.741037 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.740963    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-hpcwt"]
Nov 04 03:57:35.757651 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.754008    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 04 03:57:35.757651 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.754138    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-fqlnh\""
Nov 04 03:57:35.757651 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.754220    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 04 03:57:35.757651 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.754306    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 03:57:35.765649 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.758748    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 03:57:35.767532 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.767482    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:35.825721 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.825676    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6gr5\" (UniqueName: \"kubernetes.io/projected/9815ccf8-1a32-46c0-9037-666e183f09a9-kube-api-access-v6gr5\") pod \"ai-gateway-controller-69464886d7-hpcwt\" (UID: \"9815ccf8-1a32-46c0-9037-666e183f09a9\") " pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:57:35.826183 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.826157    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9815ccf8-1a32-46c0-9037-666e183f09a9-metrics-certs\") pod \"ai-gateway-controller-69464886d7-hpcwt\" (UID: \"9815ccf8-1a32-46c0-9037-666e183f09a9\") " pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:57:35.829378 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.829358    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9815ccf8-1a32-46c0-9037-666e183f09a9-metrics-certs\") pod \"ai-gateway-controller-69464886d7-hpcwt\" (UID: \"9815ccf8-1a32-46c0-9037-666e183f09a9\") " pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:57:35.845219 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.845198    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6gr5\" (UniqueName: \"kubernetes.io/projected/9815ccf8-1a32-46c0-9037-666e183f09a9-kube-api-access-v6gr5\") pod \"ai-gateway-controller-69464886d7-hpcwt\" (UID: \"9815ccf8-1a32-46c0-9037-666e183f09a9\") " pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:57:35.981361 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:35.981078    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:36.033252 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:36.033227    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:57:36.363661 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:36.362925    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-hpcwt"]
Nov 04 03:57:36.392654 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:36.387016    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:57:36.392654 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:36.387102    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp" podUID="90da3418-09f9-4d87-b646-75914ed6a3bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 04 03:57:36.392654 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:57:36.391142    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-476c11bae5ce3e3613aef58714913d87236996e27d2dae0758a4327873a7d295 WatchSource:0}: Error finding container 476c11bae5ce3e3613aef58714913d87236996e27d2dae0758a4327873a7d295: Status 404 returned error can't find the container with id 476c11bae5ce3e3613aef58714913d87236996e27d2dae0758a4327873a7d295
Nov 04 03:57:36.397849 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:36.397372    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:57:36.397849 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:36.397415    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" podUID="7427f8fe-4e90-426b-9e23-7054641123b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 03:57:36.687702 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:36.687133    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt" event={"ID":"9815ccf8-1a32-46c0-9037-666e183f09a9","Type":"ContainerStarted","Data":"476c11bae5ce3e3613aef58714913d87236996e27d2dae0758a4327873a7d295"}
Nov 04 03:57:37.386884 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:37.386835    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:57:37.387139 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:37.386918    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp" podUID="90da3418-09f9-4d87-b646-75914ed6a3bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 04 03:57:37.397214 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:37.397185    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:57:37.397304 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:37.397242    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" podUID="7427f8fe-4e90-426b-9e23-7054641123b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 03:57:38.387149 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:38.387111    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 04 03:57:38.387575 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:38.387179    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp" podUID="90da3418-09f9-4d87-b646-75914ed6a3bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 04 03:57:38.397524 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:38.397493    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:57:38.397664 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:38.397551    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" podUID="7427f8fe-4e90-426b-9e23-7054641123b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 03:57:39.397095 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:39.397058    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 03:57:39.397574 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:39.397112    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2" podUID="7427f8fe-4e90-426b-9e23-7054641123b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 03:57:40.288458 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:40.288424    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:40.289308 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:40.289261    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:40.296267 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:40.296231    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-xrqpp"
Nov 04 03:57:40.402264 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:40.402230    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:40.404649 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:40.403093    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:40.404649 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:40.403824    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-wjbj2"
Nov 04 03:57:41.716284 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:41.716247    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt" event={"ID":"9815ccf8-1a32-46c0-9037-666e183f09a9","Type":"ContainerStarted","Data":"7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380"}
Nov 04 03:57:41.716855 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:41.716336    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:57:41.744590 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:41.744532    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt" podStartSLOduration=2.341430237 podStartE2EDuration="6.744517093s" totalImagesPullingTime="4.403086856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:41.743910708 +0000 UTC m=+456.077848586" watchObservedRunningTime="2026-11-04 03:57:41.744517093 +0000 UTC m=+456.078454971"
Nov 04 03:57:42.136833 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.136795    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"]
Nov 04 03:57:42.143039 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.143019    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.180650 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.180583    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"]
Nov 04 03:57:42.191089 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.191062    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.191277 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.191096    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13ffed84-554f-440a-8016-20e8344e5135-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.191277 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.191191    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krxlh\" (UniqueName: \"kubernetes.io/projected/13ffed84-554f-440a-8016-20e8344e5135-kube-api-access-krxlh\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.191277 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.191224    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.191277 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.191262    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.191409 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.191330    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.191409 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.191348    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13ffed84-554f-440a-8016-20e8344e5135-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.191409 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.191367    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13ffed84-554f-440a-8016-20e8344e5135-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.191508 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.191484    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.292464 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.292419    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krxlh\" (UniqueName: \"kubernetes.io/projected/13ffed84-554f-440a-8016-20e8344e5135-kube-api-access-krxlh\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.292694 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.292485    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.292694 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.292561    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.292694 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.292600    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.292694 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.292653    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13ffed84-554f-440a-8016-20e8344e5135-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.292694 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.292683    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13ffed84-554f-440a-8016-20e8344e5135-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.292970 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.292724    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.292970 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.292797    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.292970 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.292834    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13ffed84-554f-440a-8016-20e8344e5135-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.293109 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.293042    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-credential-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.293109 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.293073    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-workload-socket\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.293234 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.293205    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-istio-data\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.293311 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.293220    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-workload-certs\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.293579 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.293552    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13ffed84-554f-440a-8016-20e8344e5135-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.294997 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.294980    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13ffed84-554f-440a-8016-20e8344e5135-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.295215 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.295196    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13ffed84-554f-440a-8016-20e8344e5135-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.305380 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.305357    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krxlh\" (UniqueName: \"kubernetes.io/projected/13ffed84-554f-440a-8016-20e8344e5135-kube-api-access-krxlh\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.306855 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.306835    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13ffed84-554f-440a-8016-20e8344e5135-istio-token\") pod \"maas-default-gateway-openshift-default-7d64fcddb7-96lb9\" (UID: \"13ffed84-554f-440a-8016-20e8344e5135\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.460485 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.460386    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:42.787011 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.786935    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"]
Nov 04 03:57:42.788246 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:57:42.788218    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ffed84_554f_440a_8016_20e8344e5135.slice/crio-418f516f3c0402d4b66e49bea2f9cad1da09ef4f9d8aa5deffe2d12cf7f3a7e0 WatchSource:0}: Error finding container 418f516f3c0402d4b66e49bea2f9cad1da09ef4f9d8aa5deffe2d12cf7f3a7e0: Status 404 returned error can't find the container with id 418f516f3c0402d4b66e49bea2f9cad1da09ef4f9d8aa5deffe2d12cf7f3a7e0
Nov 04 03:57:42.790409 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.790379    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:57:42.790467 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.790447    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:57:42.790506 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.790478    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:57:42.926069 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:42.925961    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:43.145271 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:43.145228    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:57:43.733835 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:43.733787    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" event={"ID":"13ffed84-554f-440a-8016-20e8344e5135","Type":"ContainerStarted","Data":"44e582a7feeb1912b44b7f2ddb7a198d8f2e0d5a86e23873e5c52487587b5bd3"}
Nov 04 03:57:43.734044 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:43.733900    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" event={"ID":"13ffed84-554f-440a-8016-20e8344e5135","Type":"ContainerStarted","Data":"418f516f3c0402d4b66e49bea2f9cad1da09ef4f9d8aa5deffe2d12cf7f3a7e0"}
Nov 04 03:57:43.786814 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:43.784404    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" podStartSLOduration=1.784382543 podStartE2EDuration="1.784382543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:43.783915493 +0000 UTC m=+458.117853406" watchObservedRunningTime="2026-11-04 03:57:43.784382543 +0000 UTC m=+458.118320419"
Nov 04 03:57:44.464990 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:44.464955    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:44.467097 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:44.467068    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-96lb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 04 03:57:44.467340 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:44.467300    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" podUID="13ffed84-554f-440a-8016-20e8344e5135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 04 03:57:45.464643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:45.464021    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-96lb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 04 03:57:45.464643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:45.464084    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" podUID="13ffed84-554f-440a-8016-20e8344e5135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 04 03:57:46.474766 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:46.474730    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-96lb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 04 03:57:46.475223 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:46.474915    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" podUID="13ffed84-554f-440a-8016-20e8344e5135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 04 03:57:47.466112 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:47.465650    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-96lb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 04 03:57:47.466112 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:47.465708    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" podUID="13ffed84-554f-440a-8016-20e8344e5135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 04 03:57:48.469644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:48.469026    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-96lb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 04 03:57:48.469644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:48.469091    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" podUID="13ffed84-554f-440a-8016-20e8344e5135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 04 03:57:49.461010 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:49.460973    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-96lb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 04 03:57:49.461188 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:49.461033    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" podUID="13ffed84-554f-440a-8016-20e8344e5135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 04 03:57:51.087820 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:57:51.087778    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:57:51.461927 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:51.461888    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-96lb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:57:51.462102 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:51.461967    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" podUID="13ffed84-554f-440a-8016-20e8344e5135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:57:52.503761 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:52.503724    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-96lb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:57:52.504280 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:52.503777    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" podUID="13ffed84-554f-440a-8016-20e8344e5135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:57:52.723603 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:52.723572    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:57:52.999349 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:52.999310    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-79b649855b-9p994"]
Nov 04 03:57:53.006898 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.006871    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-79b649855b-9p994"
Nov 04 03:57:53.021927 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.021905    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-79b649855b-9p994"]
Nov 04 03:57:53.130387 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.130354    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1636e316-ede4-4230-a002-46b1fcea8469-metrics-certs\") pod \"ai-gateway-controller-79b649855b-9p994\" (UID: \"1636e316-ede4-4230-a002-46b1fcea8469\") " pod="opendatahub/ai-gateway-controller-79b649855b-9p994"
Nov 04 03:57:53.130521 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.130392    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfz7l\" (UniqueName: \"kubernetes.io/projected/1636e316-ede4-4230-a002-46b1fcea8469-kube-api-access-qfz7l\") pod \"ai-gateway-controller-79b649855b-9p994\" (UID: \"1636e316-ede4-4230-a002-46b1fcea8469\") " pod="opendatahub/ai-gateway-controller-79b649855b-9p994"
Nov 04 03:57:53.231516 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.231487    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1636e316-ede4-4230-a002-46b1fcea8469-metrics-certs\") pod \"ai-gateway-controller-79b649855b-9p994\" (UID: \"1636e316-ede4-4230-a002-46b1fcea8469\") " pod="opendatahub/ai-gateway-controller-79b649855b-9p994"
Nov 04 03:57:53.231662 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.231520    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz7l\" (UniqueName: \"kubernetes.io/projected/1636e316-ede4-4230-a002-46b1fcea8469-kube-api-access-qfz7l\") pod \"ai-gateway-controller-79b649855b-9p994\" (UID: \"1636e316-ede4-4230-a002-46b1fcea8469\") " pod="opendatahub/ai-gateway-controller-79b649855b-9p994"
Nov 04 03:57:53.233834 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.233815    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1636e316-ede4-4230-a002-46b1fcea8469-metrics-certs\") pod \"ai-gateway-controller-79b649855b-9p994\" (UID: \"1636e316-ede4-4230-a002-46b1fcea8469\") " pod="opendatahub/ai-gateway-controller-79b649855b-9p994"
Nov 04 03:57:53.244380 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.244355    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfz7l\" (UniqueName: \"kubernetes.io/projected/1636e316-ede4-4230-a002-46b1fcea8469-kube-api-access-qfz7l\") pod \"ai-gateway-controller-79b649855b-9p994\" (UID: \"1636e316-ede4-4230-a002-46b1fcea8469\") " pod="opendatahub/ai-gateway-controller-79b649855b-9p994"
Nov 04 03:57:53.323789 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.323725    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-79b649855b-9p994"
Nov 04 03:57:53.544753 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.544721    6751 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d64fcddb7-96lb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:57:53.545166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.544795    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9" podUID="13ffed84-554f-440a-8016-20e8344e5135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:57:53.603776 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.603746    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-79b649855b-9p994"]
Nov 04 03:57:53.607401 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:57:53.607375    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1636e316_ede4_4230_a002_46b1fcea8469.slice/crio-1efb95b43eb3125900f863284e50ea8ae1965b0edcc0925edf1e4986a9b1fa49 WatchSource:0}: Error finding container 1efb95b43eb3125900f863284e50ea8ae1965b0edcc0925edf1e4986a9b1fa49: Status 404 returned error can't find the container with id 1efb95b43eb3125900f863284e50ea8ae1965b0edcc0925edf1e4986a9b1fa49
Nov 04 03:57:53.625807 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.625786    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:53.626074 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.626044    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:53.627396 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.627376    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d64fcddb7-96lb9"
Nov 04 03:57:53.798504 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:53.798467    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-79b649855b-9p994" event={"ID":"1636e316-ede4-4230-a002-46b1fcea8469","Type":"ContainerStarted","Data":"1efb95b43eb3125900f863284e50ea8ae1965b0edcc0925edf1e4986a9b1fa49"}
Nov 04 03:57:54.804616 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:54.804576    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-79b649855b-9p994" event={"ID":"1636e316-ede4-4230-a002-46b1fcea8469","Type":"ContainerStarted","Data":"a10765b0bd1f906c57526367363364f833f11710e6190877a58c9c47281a8381"}
Nov 04 03:57:54.805075 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:54.804763    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-79b649855b-9p994"
Nov 04 03:57:54.825339 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:57:54.825297    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-79b649855b-9p994" podStartSLOduration=2.377862332 podStartE2EDuration="2.825285951s" totalImagesPullingTime="447.423619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:57:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:57:54.8235235 +0000 UTC m=+469.157461382" watchObservedRunningTime="2026-11-04 03:57:54.825285951 +0000 UTC m=+469.159223828"
Nov 04 03:58:05.811577 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:05.811544    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-79b649855b-9p994"
Nov 04 03:58:05.865049 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:05.865015    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-hpcwt"]
Nov 04 03:58:05.865289 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:05.865244    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt" podUID="9815ccf8-1a32-46c0-9037-666e183f09a9" containerName="manager" containerID="cri-o://7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380" gracePeriod=30
Nov 04 03:58:06.115042 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.115018    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:58:06.248466 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.248435    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9815ccf8-1a32-46c0-9037-666e183f09a9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9815ccf8-1a32-46c0-9037-666e183f09a9-metrics-certs\") pod \"9815ccf8-1a32-46c0-9037-666e183f09a9\" (UID: \"9815ccf8-1a32-46c0-9037-666e183f09a9\") "
Nov 04 03:58:06.248654 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.248635    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9815ccf8-1a32-46c0-9037-666e183f09a9-kube-api-access-v6gr5\" (UniqueName: \"kubernetes.io/projected/9815ccf8-1a32-46c0-9037-666e183f09a9-kube-api-access-v6gr5\") pod \"9815ccf8-1a32-46c0-9037-666e183f09a9\" (UID: \"9815ccf8-1a32-46c0-9037-666e183f09a9\") "
Nov 04 03:58:06.250678 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.250654    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9815ccf8-1a32-46c0-9037-666e183f09a9-kube-api-access-v6gr5" pod "9815ccf8-1a32-46c0-9037-666e183f09a9" (UID: "9815ccf8-1a32-46c0-9037-666e183f09a9"). InnerVolumeSpecName "kube-api-access-v6gr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 03:58:06.250678 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.250667    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9815ccf8-1a32-46c0-9037-666e183f09a9-metrics-certs" pod "9815ccf8-1a32-46c0-9037-666e183f09a9" (UID: "9815ccf8-1a32-46c0-9037-666e183f09a9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 03:58:06.349869 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.349801    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6gr5\" (UniqueName: \"kubernetes.io/projected/9815ccf8-1a32-46c0-9037-666e183f09a9-kube-api-access-v6gr5\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:58:06.349869 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.349827    6751 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9815ccf8-1a32-46c0-9037-666e183f09a9-metrics-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 03:58:06.649875 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.649794    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:06.858128 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.858092    6751 generic.go:356] "Generic (PLEG): container finished" podID="9815ccf8-1a32-46c0-9037-666e183f09a9" containerID="7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380" exitCode=0
Nov 04 03:58:06.858525 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.858142    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt"
Nov 04 03:58:06.858525 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.858189    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt" event={"ID":"9815ccf8-1a32-46c0-9037-666e183f09a9","Type":"ContainerDied","Data":"7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380"}
Nov 04 03:58:06.858525 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.858243    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-69464886d7-hpcwt" event={"ID":"9815ccf8-1a32-46c0-9037-666e183f09a9","Type":"ContainerDied","Data":"476c11bae5ce3e3613aef58714913d87236996e27d2dae0758a4327873a7d295"}
Nov 04 03:58:06.858525 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.858269    6751 scope.go:122] "RemoveContainer" containerID="7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380"
Nov 04 03:58:06.866891 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.866715    6751 scope.go:122] "RemoveContainer" containerID="7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380"
Nov 04 03:58:06.867014 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:06.866996    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380\": container with ID starting with 7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380 not found: ID does not exist" containerID="7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380"
Nov 04 03:58:06.867071 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.867028    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380"} err="failed to get container status \"7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380\": rpc error: code = NotFound desc = could not find container \"7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380\": container with ID starting with 7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380 not found: ID does not exist"
Nov 04 03:58:06.881118 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.881093    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-hpcwt"]
Nov 04 03:58:06.885047 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:06.885026    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-hpcwt"]
Nov 04 03:58:07.207326 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:07.207290    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:07.759270 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:07.759235    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:07.783869 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:07.783841    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9815ccf8-1a32-46c0-9037-666e183f09a9" path="/var/lib/kubelet/pods/9815ccf8-1a32-46c0-9037-666e183f09a9/volumes"
Nov 04 03:58:08.279200 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:08.279162    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:08.289938 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:08.289908    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:09.933435 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:09.933398    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:10.154398 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:10.154365    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:11.498798 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:11.498762    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:11.795058 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:11.794975    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:17.252480 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:17.252443    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:21.086217 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:21.086174    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:58:23.308144 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:23.308108    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:27.262848 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:27.262816    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:34.800003 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:34.799959    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:35.033728 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.033648    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:35.150795 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.150763    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"]
Nov 04 03:58:35.151137 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.151125    6751 cpu_manager.go:414] "containerMap: removing container" podUID="9815ccf8-1a32-46c0-9037-666e183f09a9" containerName="manager"
Nov 04 03:58:35.151181 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.151139    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9815ccf8-1a32-46c0-9037-666e183f09a9" containerName="manager"
Nov 04 03:58:35.151230 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.151222    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="9815ccf8-1a32-46c0-9037-666e183f09a9" containerName="manager"
Nov 04 03:58:35.155798 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.155780    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.165357 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.165336    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-998997-openshift-default-dockercfg-sksxz\""
Nov 04 03:58:35.172951 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.172926    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"]
Nov 04 03:58:35.205297 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.205269    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-podinfo\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.205405 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.205340    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzkgt\" (UniqueName: \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-kube-api-access-hzkgt\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.205405 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.205387    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-certs\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.205526 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.205468    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.205581 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.205532    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istiod-ca-cert\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.205668 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.205593    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-credential-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.205668 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.205658    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-envoy\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.205778 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.205697    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-data\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.205821 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.205800    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-token\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.280153 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.280122    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"]
Nov 04 03:58:35.283789 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.283767    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.292696 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.292675    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-64c20d-openshift-default-dockercfg-xpqv6\""
Nov 04 03:58:35.303668 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.303647    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"]
Nov 04 03:58:35.305403 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.305368    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:35.306577 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.306555    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzkgt\" (UniqueName: \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-kube-api-access-hzkgt\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.306712 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.306628    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-certs\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.306712 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.306666    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.306712 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.306707    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istiod-ca-cert\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.306883 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.306758    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-credential-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.306883 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.306788    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-envoy\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.306883 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.306829    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-data\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.306883 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.306878    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-token\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.307077 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.306964    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-podinfo\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.307077 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.306998    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-certs\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.307178 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.307164    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-credential-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.307264 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.307241    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-socket\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.307372 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.307354    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-data\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.307881 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.307840    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istiod-ca-cert\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.309582 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.309560    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-envoy\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.310244 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.310192    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-podinfo\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.326199 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.326174    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-token\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.329129 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.329106    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzkgt\" (UniqueName: \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-kube-api-access-hzkgt\") pod \"e2e-worker-w3-998997-openshift-default-768df74449-8f6fh\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") " pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.377816 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.377785    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"]
Nov 04 03:58:35.382579 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.382559    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.401379 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.401294    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-4afe0e-openshift-default-dockercfg-x52mt\""
Nov 04 03:58:35.405444 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.405421    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"]
Nov 04 03:58:35.407792 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.407770    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.407897 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.407797    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdjl9\" (UniqueName: \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-kube-api-access-cdjl9\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.407897 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.407833    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k2lt\" (UniqueName: \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-kube-api-access-2k2lt\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.408016 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.407932    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-credential-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.408016 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.407956    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.408016 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.407973    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.408016 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.407993    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-token\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.408224 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408130    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.408224 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408148    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-data\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.408224 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408165    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-certs\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.408224 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408183    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.408224 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408203    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.408466 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408242    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-podinfo\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.408466 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408311    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.408466 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408423    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.408466 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408442    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-envoy\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.408695 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408471    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istiod-ca-cert\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.408695 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.408544    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.469152 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.469126    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:35.509951 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.509908    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.510119 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.509959    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdjl9\" (UniqueName: \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-kube-api-access-cdjl9\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.510119 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510017    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k2lt\" (UniqueName: \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-kube-api-access-2k2lt\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.510119 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510063    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-credential-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.510119 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510084    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.510343 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510122    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.510343 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510153    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-token\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.510343 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510205    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.510343 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510232    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-data\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.510343 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510261    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-certs\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.510343 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510290    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.510343 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510313    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.510343 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510339    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-podinfo\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.510791 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510369    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.510791 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510412    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.510791 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510439    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-envoy\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.510791 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510462    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istiod-ca-cert\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.510791 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.510481    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.511056 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.511034    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-data\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.511280 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.511254    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.512547 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.511976    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.512547 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.511989    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.512547 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.512234    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-certs\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.512547 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.512261    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-credential-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.512547 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.512375    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.512916 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.512572    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.512916 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.512868    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.513045 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.512981    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istiod-ca-cert\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.513486 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.513461    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-envoy\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.513567 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.513536    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.513667 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.513649    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-podinfo\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.514923 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.514906    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.559098 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.559064    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-token\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.560346 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.560322    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k2lt\" (UniqueName: \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-kube-api-access-2k2lt\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.591969 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.591931    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:35.600101 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.600072    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.641612 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.641572    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdjl9\" (UniqueName: \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-kube-api-access-cdjl9\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.701706 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.701643    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:35.766051 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.766026    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"]
Nov 04 03:58:35.771108 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.771083    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.802141 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.801954    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-36730f-openshift-default-dockercfg-czsxm\""
Nov 04 03:58:35.804415 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.804377    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"]
Nov 04 03:58:35.805058 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:58:35.805032    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb601bd1_0bb6_4c68_b5b6_5955e3763ecc.slice/crio-6377d906d271fe1c7cc78c54eb2c9a431962506313545ee27020cb9e01d51bbc WatchSource:0}: Error finding container 6377d906d271fe1c7cc78c54eb2c9a431962506313545ee27020cb9e01d51bbc: Status 404 returned error can't find the container with id 6377d906d271fe1c7cc78c54eb2c9a431962506313545ee27020cb9e01d51bbc
Nov 04 03:58:35.806450 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.806428    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"]
Nov 04 03:58:35.807811 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.807783    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:35.807910 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.807847    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:35.807910 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.807885    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:35.813875 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.813853    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-token\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.813982 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.813885    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-envoy\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.813982 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.813915    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwdrh\" (UniqueName: \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-kube-api-access-vwdrh\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.814107 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.814014    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-data\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.814107 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.814065    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-podinfo\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.814219 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.814195    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.814277 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.814260    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/561d0a19-dce6-44d7-a998-86df9faaceb5-istiod-ca-cert\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.814333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.814320    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-certs\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.814393 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.814360    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-credential-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.852396 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.852372    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:35.898688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.898659    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:35.915465 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.915434    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-token\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.915588 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.915489    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-envoy\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.915588 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.915523    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwdrh\" (UniqueName: \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-kube-api-access-vwdrh\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.915725 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.915595    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-data\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.915725 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.915672    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-podinfo\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.915830 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.915791    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.915885 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.915829    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/561d0a19-dce6-44d7-a998-86df9faaceb5-istiod-ca-cert\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.915937 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.915892    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-certs\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.915937 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.915919    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-credential-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.917453 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.916557    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-data\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.917453 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.916604    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-credential-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.917453 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.917179    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-certs\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.917453 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.917399    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/561d0a19-dce6-44d7-a998-86df9faaceb5-istiod-ca-cert\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.917453 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.917417    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-socket\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.918800 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.918763    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-envoy\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.919185 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.919166    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-podinfo\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.936235 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.936203    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwdrh\" (UniqueName: \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-kube-api-access-vwdrh\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.937163 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.937129    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-token\") pod \"e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") " pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:35.995646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.988586    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" event={"ID":"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc","Type":"ContainerStarted","Data":"d8385c0c0f04f62f88fd7a24e1e8c35a3a92e6f177c4bbb88321913a5eaa5ff1"}
Nov 04 03:58:35.995646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:35.988694    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" event={"ID":"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc","Type":"ContainerStarted","Data":"6377d906d271fe1c7cc78c54eb2c9a431962506313545ee27020cb9e01d51bbc"}
Nov 04 03:58:36.024644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.010340    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"]
Nov 04 03:58:36.025101 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.024897    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.025101 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.025040    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.025214 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.025144    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.031175 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.029940    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" podStartSLOduration=1.02992355 podStartE2EDuration="1.02992355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:36.024295684 +0000 UTC m=+510.358233562" watchObservedRunningTime="2026-11-04 03:58:36.02992355 +0000 UTC m=+510.363861429"
Nov 04 03:58:36.036823 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.036798    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"]
Nov 04 03:58:36.036948 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.036851    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"]
Nov 04 03:58:36.037016 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.036967    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.046079 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.046061    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-78ca37-openshift-default-dockercfg-84ldj\""
Nov 04 03:58:36.060659 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:36.056587    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:36.060659 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:36.057867    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:36.085462 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.085435    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:36.087595 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.087500    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:36.122465 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.122440    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-certs\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.122580 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.122476    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff9d333-98c8-4671-a057-b07ac7497812-istio-podinfo\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.122743 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.122718    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-data\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.122808 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.122760    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-credential-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.122889 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.122874    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-istio-token\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.122941 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.122901    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff9d333-98c8-4671-a057-b07ac7497812-istiod-ca-cert\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.122979 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.122964    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.123051 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.123037    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xnmp\" (UniqueName: \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-kube-api-access-5xnmp\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.123110 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.123072    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-envoy\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.231166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.230700    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-data\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.231166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.230766    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-credential-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.231166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.230839    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-istio-token\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.231166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.230867    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff9d333-98c8-4671-a057-b07ac7497812-istiod-ca-cert\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.231166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.230919    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.231166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.230951    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xnmp\" (UniqueName: \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-kube-api-access-5xnmp\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.231166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.230986    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-envoy\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.231166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.231041    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-certs\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.231166 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.231069    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff9d333-98c8-4671-a057-b07ac7497812-istio-podinfo\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.232911 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.232526    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff9d333-98c8-4671-a057-b07ac7497812-istiod-ca-cert\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.232911 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.232860    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-data\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.233146 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.233123    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-credential-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.233260 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.233234    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-socket\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.233888 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.233583    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-certs\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.234509 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.234481    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff9d333-98c8-4671-a057-b07ac7497812-istio-podinfo\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.235470 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.235449    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-envoy\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.277981 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.277934    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"]
Nov 04 03:58:36.281151 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:58:36.281124    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5931acd2_4ef3_4a72_b76f_c792ffee8c4a.slice/crio-1cf5a3b7597fb97d5ffe939031d00c2edd2f518ccccd26fbbced635e04075c74 WatchSource:0}: Error finding container 1cf5a3b7597fb97d5ffe939031d00c2edd2f518ccccd26fbbced635e04075c74: Status 404 returned error can't find the container with id 1cf5a3b7597fb97d5ffe939031d00c2edd2f518ccccd26fbbced635e04075c74
Nov 04 03:58:36.282675 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.282654    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-istio-token\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.283672 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.283649    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xnmp\" (UniqueName: \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-kube-api-access-5xnmp\") pod \"e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") " pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.284147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.284119    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.284195 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.284186    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.284235 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.284214    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.343736 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.343711    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:36.364930 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.364901    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:36.386152 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.386023    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"]
Nov 04 03:58:36.393061 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:58:36.393030    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561d0a19_dce6_44d7_a998_86df9faaceb5.slice/crio-ed95fb68eed5c7f270e127691f19ab11b80ffcfea22d2b1dca272b4e5b4139cd WatchSource:0}: Error finding container ed95fb68eed5c7f270e127691f19ab11b80ffcfea22d2b1dca272b4e5b4139cd: Status 404 returned error can't find the container with id ed95fb68eed5c7f270e127691f19ab11b80ffcfea22d2b1dca272b4e5b4139cd
Nov 04 03:58:36.396399 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.396364    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.396487 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.396452    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.396550 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.396498    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.472684 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.470141    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:36.484911 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.484654    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:36.669542 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.669515    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"]
Nov 04 03:58:36.672947 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:58:36.672914    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff9d333_98c8_4671_a057_b07ac7497812.slice/crio-6703dbc7e3223541ef5c1f8bbd52435d377b4d89d3e81863ca8916931e0cf553 WatchSource:0}: Error finding container 6703dbc7e3223541ef5c1f8bbd52435d377b4d89d3e81863ca8916931e0cf553: Status 404 returned error can't find the container with id 6703dbc7e3223541ef5c1f8bbd52435d377b4d89d3e81863ca8916931e0cf553
Nov 04 03:58:36.676388 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.676347    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.676510 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.676445    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.676510 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.676489    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:36.994842 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.994764    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" event={"ID":"5931acd2-4ef3-4a72-b76f-c792ffee8c4a","Type":"ContainerStarted","Data":"05b636eb769f7a23c2e7b48f04d36d3a24fd15bcc1a8928a27d0719dbceef0b3"}
Nov 04 03:58:36.994842 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.994816    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" event={"ID":"5931acd2-4ef3-4a72-b76f-c792ffee8c4a","Type":"ContainerStarted","Data":"1cf5a3b7597fb97d5ffe939031d00c2edd2f518ccccd26fbbced635e04075c74"}
Nov 04 03:58:36.996409 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.996376    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" event={"ID":"f8e1215b-ab44-4926-81e7-dcb61def4ad5","Type":"ContainerStarted","Data":"788e4f9880730e1022bacb655aff628f35a78e52c979213aa4321bf9c74a6eb5"}
Nov 04 03:58:36.996532 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.996437    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" event={"ID":"f8e1215b-ab44-4926-81e7-dcb61def4ad5","Type":"ContainerStarted","Data":"013bb66c380b60c7f2513dd89265d2290919863a3d2c08409d0a4c7bc0d64f54"}
Nov 04 03:58:36.998105 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.998078    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" event={"ID":"aff9d333-98c8-4671-a057-b07ac7497812","Type":"ContainerStarted","Data":"39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9"}
Nov 04 03:58:36.998244 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.998137    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" event={"ID":"aff9d333-98c8-4671-a057-b07ac7497812","Type":"ContainerStarted","Data":"6703dbc7e3223541ef5c1f8bbd52435d377b4d89d3e81863ca8916931e0cf553"}
Nov 04 03:58:36.999774 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.999752    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" event={"ID":"561d0a19-dce6-44d7-a998-86df9faaceb5","Type":"ContainerStarted","Data":"7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a"}
Nov 04 03:58:36.999869 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.999793    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" event={"ID":"561d0a19-dce6-44d7-a998-86df9faaceb5","Type":"ContainerStarted","Data":"ed95fb68eed5c7f270e127691f19ab11b80ffcfea22d2b1dca272b4e5b4139cd"}
Nov 04 03:58:36.999978 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:36.999967    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:37.000899 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.000878    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:37.021815 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.021768    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" podStartSLOduration=2.021755843 podStartE2EDuration="2.021755843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:37.019481903 +0000 UTC m=+511.353419780" watchObservedRunningTime="2026-11-04 03:58:37.021755843 +0000 UTC m=+511.355693720"
Nov 04 03:58:37.046814 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.046774    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" podStartSLOduration=2.046761671 podStartE2EDuration="2.046761671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:37.04601553 +0000 UTC m=+511.379953410" watchObservedRunningTime="2026-11-04 03:58:37.046761671 +0000 UTC m=+511.380699549"
Nov 04 03:58:37.075739 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.075696    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" podStartSLOduration=2.075680749 podStartE2EDuration="2.075680749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:37.074821312 +0000 UTC m=+511.408759191" watchObservedRunningTime="2026-11-04 03:58:37.075680749 +0000 UTC m=+511.409618630"
Nov 04 03:58:37.087780 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.087755    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:37.092393 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.092373    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:37.137307 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.137241    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" podStartSLOduration=2.137222311 podStartE2EDuration="2.137222311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:37.134756044 +0000 UTC m=+511.468693925" watchObservedRunningTime="2026-11-04 03:58:37.137222311 +0000 UTC m=+511.471160189"
Nov 04 03:58:37.301502 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:37.301388    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:37.365979 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.365947    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:37.371115 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.371091    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:37.701803 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.701766    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:37.706419 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.706394    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:37.899434 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.899406    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:37.904560 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:37.904537    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:38.014780 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:38.014689    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:38.015532 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:38.015260    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:38.015532 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:38.015301    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:38.015532 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:38.015327    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:38.015532 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:38.015378    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:58:38.015532 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:38.015419    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:38.015825 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:38.015546    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:58:38.015825 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:38.015630    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:58:38.271437 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:38.271357    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:38.415458 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:38.415430    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:38.649073 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:38.649040    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:40.042276 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:40.042246    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:40.383383 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:40.383355    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:40.627318 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:40.627284    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:41.144871 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:41.144832    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:41.370916 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:41.370883    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:41.596692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:41.596262    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:41.815692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:41.815665    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:42.404710 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:42.404659    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:42.645068 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:42.645031    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:42.856262 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:42.856234    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:43.395840 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:43.395809    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:43.617371 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:43.617340    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:43.851533 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:43.851505    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:44.056550 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.056517    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:44.416575 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.416538    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:44.416789 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.416692    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"]
Nov 04 03:58:44.420612 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.420593    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 03:58:44.430418 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.430391    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"]
Nov 04 03:58:44.459847 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.459823    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-78ca37\""
Nov 04 03:58:44.491350 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.491327    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-78ca37-dockercfg-mvrtm\""
Nov 04 03:58:44.506012 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.505986    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjn65\" (UniqueName: \"kubernetes.io/projected/f6916ba0-d2af-4ee9-8829-7d68eec86456-kube-api-access-vjn65\") pod \"payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d\" (UID: \"f6916ba0-d2af-4ee9-8829-7d68eec86456\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 03:58:44.506106 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.506026    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6916ba0-d2af-4ee9-8829-7d68eec86456-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d\" (UID: \"f6916ba0-d2af-4ee9-8829-7d68eec86456\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 03:58:44.610364 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.610324    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjn65\" (UniqueName: \"kubernetes.io/projected/f6916ba0-d2af-4ee9-8829-7d68eec86456-kube-api-access-vjn65\") pod \"payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d\" (UID: \"f6916ba0-d2af-4ee9-8829-7d68eec86456\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 03:58:44.611043 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.611014    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6916ba0-d2af-4ee9-8829-7d68eec86456-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d\" (UID: \"f6916ba0-d2af-4ee9-8829-7d68eec86456\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 03:58:44.612497 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.612062    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6916ba0-d2af-4ee9-8829-7d68eec86456-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d\" (UID: \"f6916ba0-d2af-4ee9-8829-7d68eec86456\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 03:58:44.634654 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.630279    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjn65\" (UniqueName: \"kubernetes.io/projected/f6916ba0-d2af-4ee9-8829-7d68eec86456-kube-api-access-vjn65\") pod \"payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d\" (UID: \"f6916ba0-d2af-4ee9-8829-7d68eec86456\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 03:58:44.642817 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.642789    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:44.736454 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.736380    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 03:58:44.740120 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.740009    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"]
Nov 04 03:58:44.743912 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.743893    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 03:58:44.754291 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.754233    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-36730f\""
Nov 04 03:58:44.754291 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.754274    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-36730f-dockercfg-zmzjl\""
Nov 04 03:58:44.757031 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.757005    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"]
Nov 04 03:58:44.817343 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.817302    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dv5w\" (UniqueName: \"kubernetes.io/projected/df952825-2dfe-4016-bf56-dd65d66aa8f1-kube-api-access-2dv5w\") pod \"payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s\" (UID: \"df952825-2dfe-4016-bf56-dd65d66aa8f1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 03:58:44.817496 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.817408    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df952825-2dfe-4016-bf56-dd65d66aa8f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s\" (UID: \"df952825-2dfe-4016-bf56-dd65d66aa8f1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 03:58:44.866785 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.866749    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:44.919303 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.919264    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dv5w\" (UniqueName: \"kubernetes.io/projected/df952825-2dfe-4016-bf56-dd65d66aa8f1-kube-api-access-2dv5w\") pod \"payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s\" (UID: \"df952825-2dfe-4016-bf56-dd65d66aa8f1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 03:58:44.919459 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.919310    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df952825-2dfe-4016-bf56-dd65d66aa8f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s\" (UID: \"df952825-2dfe-4016-bf56-dd65d66aa8f1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 03:58:44.920060 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.920041    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df952825-2dfe-4016-bf56-dd65d66aa8f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s\" (UID: \"df952825-2dfe-4016-bf56-dd65d66aa8f1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 03:58:44.934093 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:44.934070    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dv5w\" (UniqueName: \"kubernetes.io/projected/df952825-2dfe-4016-bf56-dd65d66aa8f1-kube-api-access-2dv5w\") pod \"payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s\" (UID: \"df952825-2dfe-4016-bf56-dd65d66aa8f1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 03:58:45.066686 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:45.061055    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"]
Nov 04 03:58:45.066686 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:58:45.065469    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6916ba0_d2af_4ee9_8829_7d68eec86456.slice/crio-0681272e814090c614e8cc4273d95a199d3071f147f209b369bd1e9b36e12523 WatchSource:0}: Error finding container 0681272e814090c614e8cc4273d95a199d3071f147f209b369bd1e9b36e12523: Status 404 returned error can't find the container with id 0681272e814090c614e8cc4273d95a199d3071f147f209b369bd1e9b36e12523
Nov 04 03:58:45.075985 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:45.075953    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:45.080260 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:45.080239    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 03:58:45.295646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:45.292198    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:45.369643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:45.367116    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"]
Nov 04 03:58:45.378417 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:58:45.378379    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf952825_2dfe_4016_bf56_dd65d66aa8f1.slice/crio-ccf115cd91737381e52679f45415d4141f582414757d5f61ffcc0aad72b966fe WatchSource:0}: Error finding container ccf115cd91737381e52679f45415d4141f582414757d5f61ffcc0aad72b966fe: Status 404 returned error can't find the container with id ccf115cd91737381e52679f45415d4141f582414757d5f61ffcc0aad72b966fe
Nov 04 03:58:45.588739 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:45.588702    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:45.819126 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:45.818480    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:46.025833 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:46.025759    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:46.136147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:46.136084    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s" event={"ID":"df952825-2dfe-4016-bf56-dd65d66aa8f1","Type":"ContainerStarted","Data":"ccf115cd91737381e52679f45415d4141f582414757d5f61ffcc0aad72b966fe"}
Nov 04 03:58:46.139043 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:46.139011    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d" event={"ID":"f6916ba0-d2af-4ee9-8829-7d68eec86456","Type":"ContainerStarted","Data":"0681272e814090c614e8cc4273d95a199d3071f147f209b369bd1e9b36e12523"}
Nov 04 03:58:46.299653 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:46.290214    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:46.995643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:46.994939    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:47.278169 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:47.278031    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:47.426702 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:47.426667    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:48.066576 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:48.066472    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:48.304465 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:48.304429    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:48.817234 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:48.817158    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:49.028614 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:49.028581    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:49.243429 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:49.243393    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:49.511302 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:49.511176    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:50.306350 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:50.306316    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:51.106365 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:51.106285    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:58:51.189722 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:51.189635    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:51.918432 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:51.918399    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:52.303164 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:52.302747    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s" event={"ID":"df952825-2dfe-4016-bf56-dd65d66aa8f1","Type":"ContainerStarted","Data":"990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d"}
Nov 04 03:58:52.303164 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:52.302832    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 03:58:52.304753 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:52.304715    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d" event={"ID":"f6916ba0-d2af-4ee9-8829-7d68eec86456","Type":"ContainerStarted","Data":"69d5554219216821498da520b7f9080138213e36115391961a7491984ec9cede"}
Nov 04 03:58:52.304898 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:52.304881    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 03:58:52.342754 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:52.342695    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s" podStartSLOduration=1.8962989019999998 podStartE2EDuration="8.342675272s" totalImagesPullingTime="6.44637637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:52.341206266 +0000 UTC m=+526.675144146" watchObservedRunningTime="2026-11-04 03:58:52.342675272 +0000 UTC m=+526.676613149"
Nov 04 03:58:52.415655 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:52.413093    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d" podStartSLOduration=1.76200057 podStartE2EDuration="8.413073603s" totalImagesPullingTime="6.651073033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:58:52.412003018 +0000 UTC m=+526.745940897" watchObservedRunningTime="2026-11-04 03:58:52.413073603 +0000 UTC m=+526.747011484"
Nov 04 03:58:53.003221 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:53.003123    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:53.043776 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:53.043726    6751 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:58:53.043946 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:53.043781    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:58:53.349447 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:53.349412    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:53.806528 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:53.806393    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:54.057204 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:54.056787    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 03:58:54.057204 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:54.056856    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 03:58:54.510644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:54.509649    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:54.711076 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:54.710993    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:55.277646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:55.277182    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:55.529703 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:55.529607    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:56.338494 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:56.338400    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:56.991923 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:56.991833    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:57.237258 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:57.237231    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:57.443119 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:58:57.443090    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815ccf8_1a32_46c0_9037_666e183f09a9.slice/crio-conmon-7d10c20fe199d55dfa60538f8fdc3492dee08cf005362f440364bbc57569c380.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 03:58:58.145772 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.145723    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:58.361477 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.361447    6751 generic.go:356] "Generic (PLEG): container finished" podID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerID="788e4f9880730e1022bacb655aff628f35a78e52c979213aa4321bf9c74a6eb5" exitCode=0
Nov 04 03:58:58.361692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.361527    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" event={"ID":"f8e1215b-ab44-4926-81e7-dcb61def4ad5","Type":"ContainerDied","Data":"788e4f9880730e1022bacb655aff628f35a78e52c979213aa4321bf9c74a6eb5"}
Nov 04 03:58:58.362017 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.361992    6751 scope.go:122] "RemoveContainer" containerID="788e4f9880730e1022bacb655aff628f35a78e52c979213aa4321bf9c74a6eb5"
Nov 04 03:58:58.363714 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.363676    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:58.363792 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.363764    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:58.363845 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.363807    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:58.365274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.365253    6751 generic.go:356] "Generic (PLEG): container finished" podID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerID="d8385c0c0f04f62f88fd7a24e1e8c35a3a92e6f177c4bbb88321913a5eaa5ff1" exitCode=0
Nov 04 03:58:58.365369 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.365282    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" event={"ID":"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc","Type":"ContainerDied","Data":"d8385c0c0f04f62f88fd7a24e1e8c35a3a92e6f177c4bbb88321913a5eaa5ff1"}
Nov 04 03:58:58.365852 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.365834    6751 scope.go:122] "RemoveContainer" containerID="d8385c0c0f04f62f88fd7a24e1e8c35a3a92e6f177c4bbb88321913a5eaa5ff1"
Nov 04 03:58:58.367229 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.367202    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:58.367384 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.367274    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:58.367384 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.367309    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:58:58.554240 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:58.554183    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:59.027029 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.009961    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:59.221999 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.221813    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:59.412759 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.412715    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" event={"ID":"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc","Type":"ContainerStarted","Data":"9b38427be7daa1244daae3f815dfd87a1a62c0eb5f5c9bb4613ae3da13469804"}
Nov 04 03:58:59.416058 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.415473    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" event={"ID":"f8e1215b-ab44-4926-81e7-dcb61def4ad5","Type":"ContainerStarted","Data":"7c7d40671b21d414f3da72fca474ae3628f154a8dc6500c2c618c1e6c0a6ba8e"}
Nov 04 03:58:59.461646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.449938    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:58:59.473644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.472033    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:58:59.473644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.472160    6751 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 04 03:58:59.473644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.472208    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 04 03:58:59.703697 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.702819    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:58:59.703697 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.703352    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 04 03:58:59.703697 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.703421    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 04 03:58:59.772232 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:58:59.772052    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:00.036828 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:00.036726    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:00.285039 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:00.284871    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:00.474494 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:00.474037    6751 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 04 03:59:00.474494 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:00.474110    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 04 03:59:00.612819 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:00.612721    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:00.702180 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:00.702107    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 04 03:59:00.702371 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:00.702181    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 04 03:59:01.196596 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:01.196487    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:01.483375 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:01.482748    6751 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 04 03:59:01.483375 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:01.482810    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 04 03:59:01.703010 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:01.702967    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 04 03:59:01.703216 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:01.703038    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 04 03:59:02.215336 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.215226    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:02.434687 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.434443    6751 generic.go:356] "Generic (PLEG): container finished" podID="aff9d333-98c8-4671-a057-b07ac7497812" containerID="39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9" exitCode=0
Nov 04 03:59:02.434687 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.434491    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" event={"ID":"aff9d333-98c8-4671-a057-b07ac7497812","Type":"ContainerDied","Data":"39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9"}
Nov 04 03:59:02.435163 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.435141    6751 scope.go:122] "RemoveContainer" containerID="39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9"
Nov 04 03:59:02.436446 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.436416    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:02.436512 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.436484    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:02.436582 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.436517    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:02.491689 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.472153    6751 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 04 03:59:02.491689 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.472216    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 04 03:59:02.504211 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.504174    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"]
Nov 04 03:59:02.522573 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.521722    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"]
Nov 04 03:59:02.522573 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.521851    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.527466 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.527359    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-998997\"/\"kube-root-ca.crt\""
Nov 04 03:59:02.527806 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.527738    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-998997\"/\"openshift-service-ca.crt\""
Nov 04 03:59:02.527924 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.527869    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-998997\"/\"default-dockercfg-jqfqv\""
Nov 04 03:59:02.528125 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.528064    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-998997\"/\"simulator-w3-998997-kserve-self-signed-certs\""
Nov 04 03:59:02.620151 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.619971    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tmp-dir\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.620830 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.620125    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tls-certs\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.620830 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.620422    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-model-cache\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.620830 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.620495    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-dshm\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.620830 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.620552    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b98qv\" (UniqueName: \"kubernetes.io/projected/03acbcbe-3995-4e05-bf6e-8430b273e0b7-kube-api-access-b98qv\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.620830 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.620670    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-home\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.631653 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.631596    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:02.705516 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.705030    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 04 03:59:02.705516 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.705095    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 04 03:59:02.731001 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.730964    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tmp-dir\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.731264 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.731245    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tls-certs\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.731461 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.731444    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-model-cache\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.731693 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.731674    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-dshm\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.731892 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.731860    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b98qv\" (UniqueName: \"kubernetes.io/projected/03acbcbe-3995-4e05-bf6e-8430b273e0b7-kube-api-access-b98qv\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.732095 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.732080    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-home\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.732775 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.732755    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-home\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.733251 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.733234    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tmp-dir\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.734441 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.734423    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-model-cache\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.742275 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.742179    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tls-certs\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.756645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.752911    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-dshm\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.781639 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.779415    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b98qv\" (UniqueName: \"kubernetes.io/projected/03acbcbe-3995-4e05-bf6e-8430b273e0b7-kube-api-access-b98qv\") pod \"simulator-w3-998997-kserve-6bbb58c68f-lc7nj\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") " pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:02.849300 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:02.849086    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:03.139785 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.139707    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:03.280651 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.277798    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"]
Nov 04 03:59:03.318317 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.318243    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 03:59:03.318477 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.318321    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 03:59:03.322487 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:59:03.322453    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03acbcbe_3995_4e05_bf6e_8430b273e0b7.slice/crio-46d398cbd8e20d053a99dde8d6f85a4cdda0421131c7da7a74f37536d3688195 WatchSource:0}: Error finding container 46d398cbd8e20d053a99dde8d6f85a4cdda0421131c7da7a74f37536d3688195: Status 404 returned error can't find the container with id 46d398cbd8e20d053a99dde8d6f85a4cdda0421131c7da7a74f37536d3688195
Nov 04 03:59:03.468465 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.449873    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" event={"ID":"aff9d333-98c8-4671-a057-b07ac7497812","Type":"ContainerStarted","Data":"f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490"}
Nov 04 03:59:03.494651 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.481068    6751 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 04 03:59:03.494651 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.481128    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 04 03:59:03.537912 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.537874    6751 generic.go:356] "Generic (PLEG): container finished" podID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerID="7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a" exitCode=0
Nov 04 03:59:03.538304 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.538141    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" event={"ID":"561d0a19-dce6-44d7-a998-86df9faaceb5","Type":"ContainerDied","Data":"7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a"}
Nov 04 03:59:03.539045 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.539023    6751 scope.go:122] "RemoveContainer" containerID="7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a"
Nov 04 03:59:03.542096 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.542068    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj" event={"ID":"03acbcbe-3995-4e05-bf6e-8430b273e0b7","Type":"ContainerStarted","Data":"46d398cbd8e20d053a99dde8d6f85a4cdda0421131c7da7a74f37536d3688195"}
Nov 04 03:59:03.542213 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.542137    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:03.542274 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.542226    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:03.542333 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.542276    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:03.545400 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.545361    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:03.708000 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.707531    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 04 03:59:03.708000 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.707589    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 04 03:59:03.823648 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:03.804375    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:04.285699 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.276386    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:04.382645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.369262    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:59:04.382645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.370420    6751 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:59:04.382645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.370471    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 03:59:04.484648 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.473544    6751 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 04 03:59:04.484648 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.474109    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 04 03:59:04.653646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.632970    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" event={"ID":"561d0a19-dce6-44d7-a998-86df9faaceb5","Type":"ContainerStarted","Data":"98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e"}
Nov 04 03:59:04.693596 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.693552    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj" event={"ID":"03acbcbe-3995-4e05-bf6e-8430b273e0b7","Type":"ContainerStarted","Data":"55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34"}
Nov 04 03:59:04.708536 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.707833    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 04 03:59:04.708536 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.707899    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 04 03:59:04.849571 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.849533    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:04.890667 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:04.890609    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:05.090044 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.089957    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:59:05.101789 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.101755    6751 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 04 03:59:05.102988 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.102956    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 04 03:59:05.103841 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.103797    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:05.338320 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.338250    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:05.370649 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.370148    6751 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:59:05.370649 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.370240    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 03:59:05.487981 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.487946    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:59:05.490522 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.490500    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:59:05.509641 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.507897    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 03:59:05.535671 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.534522    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj" podStartSLOduration=3.534503349 podStartE2EDuration="3.534503349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:04.776146846 +0000 UTC m=+539.110084725" watchObservedRunningTime="2026-11-04 03:59:05.534503349 +0000 UTC m=+539.868441223"
Nov 04 03:59:05.548103 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.548069    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:05.880643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.879229    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:05.944440 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.943566    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:59:05.944440 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.944256    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:59:05.945523 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:05.945476    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 03:59:06.102044 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:06.102007    6751 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 04 03:59:06.102311 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:06.102282    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 04 03:59:06.384656 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:06.366028    6751 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:59:06.384656 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:06.366091    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 03:59:06.494657 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:06.489137    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:06.786724 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:06.776031    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:07.088109 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:07.088016    6751 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 04 03:59:07.088364 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:07.088334    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 04 03:59:07.369454 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:07.365436    6751 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:59:07.369454 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:07.365495    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 03:59:07.657718 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:07.656730    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:08.015701 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:08.014818    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:59:08.018647 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:08.016289    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:59:08.091152 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:08.091118    6751 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 04 03:59:08.091430 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:08.091385    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 04 03:59:08.212645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:08.211365    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:08.371653 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:08.371018    6751 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:59:08.371653 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:08.371086    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 03:59:09.031143 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:09.031108    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:09.088066 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:09.088024    6751 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 04 03:59:09.088349 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:09.088318    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 04 03:59:09.237645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:09.236729    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:09.371644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:09.370059    6751 patch_prober.go:29] interesting pod/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 03:59:09.371644 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:09.370141    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 03:59:09.691697 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:09.690091    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:10.088728 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:10.088653    6751 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 04 03:59:10.089084 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:10.088720    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 04 03:59:10.377240 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:10.377209    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:59:10.378310 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:10.378289    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 03:59:10.383733 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:10.383704    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:11.088236 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:11.088198    6751 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Nov 04 03:59:11.088413 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:11.088253    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Nov 04 03:59:11.198407 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:11.198368    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:11.975603 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:11.975570    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:12.093055 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:12.093023    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:59:12.094051 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:12.094028    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 03:59:12.849475 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:12.849437    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:12.862094 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:12.862048    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:13.218339 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:13.218240    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:13.864103 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:13.864061    6751 generic.go:356] "Generic (PLEG): container finished" podID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerID="05b636eb769f7a23c2e7b48f04d36d3a24fd15bcc1a8928a27d0719dbceef0b3" exitCode=0
Nov 04 03:59:13.864692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:13.864147    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" event={"ID":"5931acd2-4ef3-4a72-b76f-c792ffee8c4a","Type":"ContainerDied","Data":"05b636eb769f7a23c2e7b48f04d36d3a24fd15bcc1a8928a27d0719dbceef0b3"}
Nov 04 03:59:13.864765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:13.864742    6751 scope.go:122] "RemoveContainer" containerID="05b636eb769f7a23c2e7b48f04d36d3a24fd15bcc1a8928a27d0719dbceef0b3"
Nov 04 03:59:13.866088 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:13.866062    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:13.866190 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:13.866145    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:13.866190 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:13.866184    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:13.869401 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:13.869216    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 03:59:13.917952 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:13.917929    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:14.485094 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:14.485041    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:14.827365 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:14.827243    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:14.879877 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:14.879838    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" event={"ID":"5931acd2-4ef3-4a72-b76f-c792ffee8c4a","Type":"ContainerStarted","Data":"beae82eb790725813b6d8222835e0118e325a7f59ffb3ecfa6be97f6b882aa0d"}
Nov 04 03:59:14.898993 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:14.898971    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:59:14.899253 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:14.899233    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 04 03:59:14.899295 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:14.899277    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 04 03:59:15.603466 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:15.603432    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:15.899145 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:15.899072    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 04 03:59:15.899145 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:15.899126    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 04 03:59:15.981673 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:15.965830    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"]
Nov 04 03:59:15.985524 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:15.985485    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"]
Nov 04 03:59:15.985729 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:15.985594    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:15.992176 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:15.992126    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-6a9cdb\"/\"kube-root-ca.crt\""
Nov 04 03:59:15.992176 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:15.992154    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-6a9cdb\"/\"simulator-w6-6a9cdb-kserve-self-signed-certs\""
Nov 04 03:59:15.992176 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:15.992130    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-6a9cdb\"/\"default-dockercfg-knx22\""
Nov 04 03:59:15.992413 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:15.992234    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-6a9cdb\"/\"openshift-service-ca.crt\""
Nov 04 03:59:16.087496 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.087467    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-model-cache\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.087692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.087502    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-home\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.087692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.087530    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-tmp-dir\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.087692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.087652    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lllsk\" (UniqueName: \"kubernetes.io/projected/59151ae2-ff43-45f0-a6bf-b5578532ce57-kube-api-access-lllsk\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.087809 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.087701    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-dshm\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.087848 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.087804    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59151ae2-ff43-45f0-a6bf-b5578532ce57-tls-certs\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.138060 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.138013    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:16.207692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.197857    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59151ae2-ff43-45f0-a6bf-b5578532ce57-tls-certs\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.207692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.197960    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-model-cache\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.207692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.197998    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-home\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.207692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.198053    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-tmp-dir\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.207692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.198113    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lllsk\" (UniqueName: \"kubernetes.io/projected/59151ae2-ff43-45f0-a6bf-b5578532ce57-kube-api-access-lllsk\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.207692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.198178    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-dshm\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.207692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.199567    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-tmp-dir\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.207692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.200368    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-home\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.207692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.200670    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-model-cache\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.222870 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.222839    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lllsk\" (UniqueName: \"kubernetes.io/projected/59151ae2-ff43-45f0-a6bf-b5578532ce57-kube-api-access-lllsk\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.223090 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.223016    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59151ae2-ff43-45f0-a6bf-b5578532ce57-tls-certs\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.230140 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.230106    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-dshm\") pod \"simulator-w6-6a9cdb-kserve-6964c84688-n8pvs\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") " pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.302281 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.302243    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:16.380232 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.378832    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:16.675691 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.675657    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"]
Nov 04 03:59:16.690774 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:59:16.690749    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59151ae2_ff43_45f0_a6bf_b5578532ce57.slice/crio-129747529f008d3412182f273a53f0338366c14f8937767e7ed4252bf8854bdf WatchSource:0}: Error finding container 129747529f008d3412182f273a53f0338366c14f8937767e7ed4252bf8854bdf: Status 404 returned error can't find the container with id 129747529f008d3412182f273a53f0338366c14f8937767e7ed4252bf8854bdf
Nov 04 03:59:16.892247 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.892135    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs" event={"ID":"59151ae2-ff43-45f0-a6bf-b5578532ce57","Type":"ContainerStarted","Data":"b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd"}
Nov 04 03:59:16.892247 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.892181    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs" event={"ID":"59151ae2-ff43-45f0-a6bf-b5578532ce57","Type":"ContainerStarted","Data":"129747529f008d3412182f273a53f0338366c14f8937767e7ed4252bf8854bdf"}
Nov 04 03:59:16.899973 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.899938    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 04 03:59:16.900349 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.899995    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 04 03:59:16.935785 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.935693    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"]
Nov 04 03:59:16.941903 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.941874    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:16.992679 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.966813    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs" podStartSLOduration=1.966794663 podStartE2EDuration="1.966794663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:16.940153664 +0000 UTC m=+551.274091542" watchObservedRunningTime="2026-11-04 03:59:16.966794663 +0000 UTC m=+551.300732543"
Nov 04 03:59:16.992679 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.985316    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-64c20d\"/\"kube-root-ca.crt\""
Nov 04 03:59:16.992679 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.989605    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-64c20d\"/\"simulator-w4-64c20d-kserve-self-signed-certs\""
Nov 04 03:59:16.992679 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.990076    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-64c20d\"/\"openshift-service-ca.crt\""
Nov 04 03:59:16.992679 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.990202    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-64c20d\"/\"default-dockercfg-5mf6f\""
Nov 04 03:59:16.994295 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:16.994270    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"]
Nov 04 03:59:17.131643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.131403    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-home\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.131643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.131468    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/992641c2-8001-44ff-a1b7-20401c15780c-tls-certs\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.131643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.131514    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-dshm\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.131643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.131591    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-tmp-dir\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.132032 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.131686    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-model-cache\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.132032 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.131832    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjvxk\" (UniqueName: \"kubernetes.io/projected/992641c2-8001-44ff-a1b7-20401c15780c-kube-api-access-gjvxk\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.137690 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.136636    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:17.232607 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.232533    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjvxk\" (UniqueName: \"kubernetes.io/projected/992641c2-8001-44ff-a1b7-20401c15780c-kube-api-access-gjvxk\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.232607 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.232592    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-home\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.232845 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.232640    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/992641c2-8001-44ff-a1b7-20401c15780c-tls-certs\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.232845 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.232667    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-dshm\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.232845 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.232723    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-tmp-dir\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.232845 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.232794    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-model-cache\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.233124 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.233099    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-home\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.233201 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.233133    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-tmp-dir\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.233201 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.233157    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-model-cache\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.234993 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.234972    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-dshm\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.235279 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.235261    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/992641c2-8001-44ff-a1b7-20401c15780c-tls-certs\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.277043 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.276989    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjvxk\" (UniqueName: \"kubernetes.io/projected/992641c2-8001-44ff-a1b7-20401c15780c-kube-api-access-gjvxk\") pod \"simulator-w4-64c20d-kserve-59c486cfbb-d5pfx\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") " pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.299056 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.299034    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:17.305121 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.305102    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:17.394974 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.394932    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:17.631361 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.631333    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"]
Nov 04 03:59:17.633879 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:59:17.633852    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod992641c2_8001_44ff_a1b7_20401c15780c.slice/crio-2fe751cb4e122de756c5d7174e6f4750846754c50dec7dd4f91dc26a0c9ba0f2 WatchSource:0}: Error finding container 2fe751cb4e122de756c5d7174e6f4750846754c50dec7dd4f91dc26a0c9ba0f2: Status 404 returned error can't find the container with id 2fe751cb4e122de756c5d7174e6f4750846754c50dec7dd4f91dc26a0c9ba0f2
Nov 04 03:59:17.898400 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.898311    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx" event={"ID":"992641c2-8001-44ff-a1b7-20401c15780c","Type":"ContainerStarted","Data":"cb6aa677ad0f7280720486dadcfde38e9a5be7c9e6ee60d54cc18d9e32574a32"}
Nov 04 03:59:17.898400 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.898357    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx" event={"ID":"992641c2-8001-44ff-a1b7-20401c15780c","Type":"ContainerStarted","Data":"2fe751cb4e122de756c5d7174e6f4750846754c50dec7dd4f91dc26a0c9ba0f2"}
Nov 04 03:59:17.898963 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.898943    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 04 03:59:17.899003 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.898988    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 04 03:59:17.922533 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.922484    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx" podStartSLOduration=1.9224714889999999 podStartE2EDuration="1.922471489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:17.921092206 +0000 UTC m=+552.255030085" watchObservedRunningTime="2026-11-04 03:59:17.922471489 +0000 UTC m=+552.256409366"
Nov 04 03:59:17.949084 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.949052    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"]
Nov 04 03:59:17.978645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.975952    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:17.978645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.977874    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"]
Nov 04 03:59:17.981080 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.981039    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-36730f\"/\"default-dockercfg-tqchj\""
Nov 04 03:59:17.981492 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.981469    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-36730f\"/\"openshift-service-ca.crt\""
Nov 04 03:59:17.981604 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.981504    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-36730f\"/\"kube-root-ca.crt\""
Nov 04 03:59:17.983337 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:17.983318    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-36730f\"/\"simulator-w1-36730f-kserve-self-signed-certs\""
Nov 04 03:59:18.045832 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.045806    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-dshm\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.045926 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.045855    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tls-certs\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.045926 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.045879    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tmp-dir\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.045926 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.045898    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-home\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.046139 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.046124    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z98wv\" (UniqueName: \"kubernetes.io/projected/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-kube-api-access-z98wv\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.046210 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.046161    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-model-cache\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.093397 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.093363    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:18.178688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.168547    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-dshm\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.178688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.169326    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tls-certs\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.178688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.169366    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tmp-dir\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.178688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.169455    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-home\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.178688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.169578    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z98wv\" (UniqueName: \"kubernetes.io/projected/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-kube-api-access-z98wv\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.178688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.174034    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tmp-dir\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.178688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.176509    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-home\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.178688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.176904    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-model-cache\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.178688 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.177481    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-model-cache\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.200643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.196138    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tls-certs\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.203513 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.203482    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z98wv\" (UniqueName: \"kubernetes.io/projected/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-kube-api-access-z98wv\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.203967 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.203947    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-dshm\") pod \"simulator-w1-36730f-kserve-7fdc4856c9-vpnm9\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") " pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.292467 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.292438    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:18.299791 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.299766    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:18.303639 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.303601    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:18.572534 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.570186    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"]
Nov 04 03:59:18.579223 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:59:18.578630    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e7f9bbb_441c_41cf_83c4_ae7d565b690b.slice/crio-39cc2e73d28228577c241664cb6363bc204adf04395eff5cc850802a5eb8970e WatchSource:0}: Error finding container 39cc2e73d28228577c241664cb6363bc204adf04395eff5cc850802a5eb8970e: Status 404 returned error can't find the container with id 39cc2e73d28228577c241664cb6363bc204adf04395eff5cc850802a5eb8970e
Nov 04 03:59:18.898986 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.898951    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 04 03:59:18.899157 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.899006    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 04 03:59:18.901199 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.901172    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:18.905280 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.905254    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9" event={"ID":"8e7f9bbb-441c-41cf-83c4-ae7d565b690b","Type":"ContainerStarted","Data":"bde4b37f2a44ed03f3ed9e8e0cbfc4b114936f9aef17858551f7cfefae4a17ee"}
Nov 04 03:59:18.905404 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.905295    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9" event={"ID":"8e7f9bbb-441c-41cf-83c4-ae7d565b690b","Type":"ContainerStarted","Data":"39cc2e73d28228577c241664cb6363bc204adf04395eff5cc850802a5eb8970e"}
Nov 04 03:59:18.925120 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:18.925079    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9" podStartSLOduration=1.925066852 podStartE2EDuration="1.925066852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:18.924077297 +0000 UTC m=+553.258015181" watchObservedRunningTime="2026-11-04 03:59:18.925066852 +0000 UTC m=+553.259004746"
Nov 04 03:59:19.293107 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:19.293013    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:19.721810 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:19.721775    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:20.000948 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:20.000854    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:59:20.001467 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:20.001375    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:59:20.002334 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:20.002311    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 03:59:20.407362 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:20.407328    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:21.086693 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:59:21.086658    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:59:21.216880 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:21.216846    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:21.894832 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:21.894798    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:22.408321 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:22.407164    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:22.704458 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:22.704390    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:23.728841 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:23.728804    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:24.788588 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:24.788557    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:25.606342 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:25.606310    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:26.305801 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:26.305770    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:26.318763 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:26.318741    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:26.407967 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:26.407928    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:26.955050 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:26.955020    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 03:59:27.113175 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:27.112636    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:27.299990 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:27.299913    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:27.312537 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:27.312513    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:27.895397 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:27.895363    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:27.961291 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:27.961265    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 03:59:28.293370 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:28.293289    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:28.305976 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:28.305953    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:28.617394 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:28.617363    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:28.966967 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:28.966902    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 03:59:29.775883 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:29.775851    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:30.483336 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:30.483297    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:31.184717 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:31.184670    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:31.890744 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:31.890699    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:32.598539 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:32.598503    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:32.983293 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:32.983224    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:33.398009 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:33.397979    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:34.188336 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:34.188303    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:35.195498 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:35.195461    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:35.690562 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:35.690529    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:35.980668 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:35.980531    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:36.492418 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:36.492385    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:37.396420 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:37.396386    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:38.203765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:38.203733    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:38.993501 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:38.993465    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:39.796668 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:39.796634    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:39.999289 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:39.999253    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:40.496332 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:40.496289    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:41.284195 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:41.284165    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:42.144749 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:42.144711    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:42.873643 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:42.873600    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:43.586690 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:43.586597    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:44.305535 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:44.305496    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:45.245132 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:45.245102    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:46.263542 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:46.263508    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:47.128838 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:47.128805    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:48.010645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.010602    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:48.307249 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.307175    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"]
Nov 04 03:59:48.310582 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.310565    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.326862 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.326840    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-6a9cdb\"/\"premium-w6-6a9cdb-kserve-self-signed-certs\""
Nov 04 03:59:48.328689 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.328663    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"]
Nov 04 03:59:48.458281 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.458238    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:48.478867 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.478464    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-home\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.478867 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.478536    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tls-certs\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.478867 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.478610    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsczq\" (UniqueName: \"kubernetes.io/projected/b457c8a5-fc7a-426d-807e-68e90ee6d97c-kube-api-access-zsczq\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.478867 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.478699    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-dshm\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.479204 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.478878    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tmp-dir\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.479204 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.478931    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-model-cache\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.580583 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.580495    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-home\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.580583 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.580548    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tls-certs\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.580827 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.580604    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsczq\" (UniqueName: \"kubernetes.io/projected/b457c8a5-fc7a-426d-807e-68e90ee6d97c-kube-api-access-zsczq\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.580827 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.580689    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-dshm\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.580827 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.580751    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tmp-dir\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.580827 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.580794    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-model-cache\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.581032 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.580926    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-home\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.581091 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.581071    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-model-cache\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.581259 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.581234    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tmp-dir\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.583107 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.583082    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-dshm\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.583218 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.583188    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tls-certs\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.594923 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.594900    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsczq\" (UniqueName: \"kubernetes.io/projected/b457c8a5-fc7a-426d-807e-68e90ee6d97c-kube-api-access-zsczq\") pod \"premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") " pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.623520 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.623496    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:48.771836 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.771800    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:48.898184 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:48.898163    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"]
Nov 04 03:59:48.899976 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:59:48.899950    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb457c8a5_fc7a_426d_807e_68e90ee6d97c.slice/crio-06869c054e135a3823d3ba5ad09f52e6b4ea02718df2857f1b04cce76bc974df WatchSource:0}: Error finding container 06869c054e135a3823d3ba5ad09f52e6b4ea02718df2857f1b04cce76bc974df: Status 404 returned error can't find the container with id 06869c054e135a3823d3ba5ad09f52e6b4ea02718df2857f1b04cce76bc974df
Nov 04 03:59:49.096902 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:49.096864    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw" event={"ID":"b457c8a5-fc7a-426d-807e-68e90ee6d97c","Type":"ContainerStarted","Data":"b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b"}
Nov 04 03:59:49.097278 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:49.096915    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw" event={"ID":"b457c8a5-fc7a-426d-807e-68e90ee6d97c","Type":"ContainerStarted","Data":"06869c054e135a3823d3ba5ad09f52e6b4ea02718df2857f1b04cce76bc974df"}
Nov 04 03:59:49.117326 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:49.117274    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw" podStartSLOduration=1.117260726 podStartE2EDuration="1.117260726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:49.116437148 +0000 UTC m=+583.450375027" watchObservedRunningTime="2026-11-04 03:59:49.117260726 +0000 UTC m=+583.451198605"
Nov 04 03:59:49.503087 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:49.503055    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:49.624492 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:49.624463    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:50.062262 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.062232    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"]
Nov 04 03:59:50.066025 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.066000    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.074937 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.074909    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-36730f\"/\"premium-w1-36730f-kserve-self-signed-certs\""
Nov 04 03:59:50.077671 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.077648    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"]
Nov 04 03:59:50.204557 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.204519    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:50.215645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.210580    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-dshm\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.215645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.210732    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-model-cache\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.215645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.210777    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8dgp\" (UniqueName: \"kubernetes.io/projected/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-kube-api-access-h8dgp\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.215645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.210836    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tls-certs\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.215645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.210882    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-home\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.215645 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.210991    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tmp-dir\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.312610 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.312531    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tmp-dir\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.312610 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.312583    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-dshm\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.312866 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.312705    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-model-cache\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.312866 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.312738    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8dgp\" (UniqueName: \"kubernetes.io/projected/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-kube-api-access-h8dgp\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.312866 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.312792    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tls-certs\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.312866 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.312853    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-home\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.313330 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.313301    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-home\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.313474 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.313338    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-model-cache\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.313474 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.313407    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tmp-dir\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.315406 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.315383    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-dshm\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.315698 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.315677    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tls-certs\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.328202 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.328173    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8dgp\" (UniqueName: \"kubernetes.io/projected/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-kube-api-access-h8dgp\") pod \"premium-w1-36730f-kserve-756c8bb874-h77mc\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") " pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.379155 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.379137    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:50.408256 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.408231    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:50.616497 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.616403    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"]
Nov 04 03:59:50.620408 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.620386    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.624328 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.624302    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-56a44f-openshift-default-dockercfg-7cnql\""
Nov 04 03:59:50.641125 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.641095    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"]
Nov 04 03:59:50.672474 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.672450    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"]
Nov 04 03:59:50.674646 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:59:50.674600    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0afc69ba_a227_4c0a_8cd4_323e7da7f9b4.slice/crio-1dd3ef1c5821110d7665528e43dc71d40d0d77f9a38743c04f4e6afd2d0d19f8 WatchSource:0}: Error finding container 1dd3ef1c5821110d7665528e43dc71d40d0d77f9a38743c04f4e6afd2d0d19f8: Status 404 returned error can't find the container with id 1dd3ef1c5821110d7665528e43dc71d40d0d77f9a38743c04f4e6afd2d0d19f8
Nov 04 03:59:50.819093 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.819053    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.819232 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.819120    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a174f8b4-9d90-407f-8914-107ddd97d3db-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.819232 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.819190    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.819232 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.819226    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.819396 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.819264    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.819396 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.819387    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.819493 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.819430    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.819534 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.819498    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.819534 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.819522    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfkmx\" (UniqueName: \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-kube-api-access-dfkmx\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921015 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.920937    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921162 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921014    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a174f8b4-9d90-407f-8914-107ddd97d3db-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921162 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921041    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921162 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921067    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921162 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921130    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921370 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921174    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921370 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921215    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921714 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921481    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921714 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921520    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921714 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921557    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfkmx\" (UniqueName: \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-kube-api-access-dfkmx\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921714 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921641    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921714 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921662    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921987 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921824    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.921987 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.921900    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a174f8b4-9d90-407f-8914-107ddd97d3db-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.923764 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.923745    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.923898 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.923875    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.933350 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.933243    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.933750 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.933730    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfkmx\" (UniqueName: \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-kube-api-access-dfkmx\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:50.938462 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:50.938441    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:51.007103 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.007073    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"]
Nov 04 03:59:51.012039 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.012006    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.020599 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.020558    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-6a9cdb-openshift-default-dockercfg-t8tng\""
Nov 04 03:59:51.023594 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.023190    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"]
Nov 04 03:59:51.111591 ip-10-0-129-253 kubenswrapper[6751]: E1104 03:59:51.111555    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 03:59:51.112964 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.112928    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:51.115541 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.115507    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc" event={"ID":"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4","Type":"ContainerStarted","Data":"1f9b57142ffd9b626b74f658a8b9656779854fc8208b92737562946756898a9a"}
Nov 04 03:59:51.115707 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.115559    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc" event={"ID":"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4","Type":"ContainerStarted","Data":"1dd3ef1c5821110d7665528e43dc71d40d0d77f9a38743c04f4e6afd2d0d19f8"}
Nov 04 03:59:51.126872 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.126848    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-credential-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.126999 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.126902    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-podinfo\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.126999 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.126976    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-data\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.127077 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.127023    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-envoy\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.127077 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.127045    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istiod-ca-cert\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.127077 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.127061    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nclb\" (UniqueName: \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-kube-api-access-2nclb\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.127163 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.127131    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-certs\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.127395 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.127380    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.127503 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.127488    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-token\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.137732 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.137688    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc" podStartSLOduration=1.137676498 podStartE2EDuration="1.137676498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:51.135031535 +0000 UTC m=+585.468969405" watchObservedRunningTime="2026-11-04 03:59:51.137676498 +0000 UTC m=+585.471614403"
Nov 04 03:59:51.224587 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.224558    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"]
Nov 04 03:59:51.226078 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:59:51.226053    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda174f8b4_9d90_407f_8914_107ddd97d3db.slice/crio-529d33cdd59b6663023e0d71633314dace0a2ac825d933daf19eb38bcce73d9e WatchSource:0}: Error finding container 529d33cdd59b6663023e0d71633314dace0a2ac825d933daf19eb38bcce73d9e: Status 404 returned error can't find the container with id 529d33cdd59b6663023e0d71633314dace0a2ac825d933daf19eb38bcce73d9e
Nov 04 03:59:51.228009 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.227985    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-token\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228076 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228038    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-credential-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228130 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228082    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-podinfo\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228178 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228158    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-data\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228421 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228396    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-envoy\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228505 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228444    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-credential-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228505 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228455    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istiod-ca-cert\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228505 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228481    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nclb\" (UniqueName: \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-kube-api-access-2nclb\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228544    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-certs\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228556    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-data\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228604    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.228692 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228663    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:51.228906 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228744    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:51.228906 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228801    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:51.229015 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.228963    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-socket\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.229015 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.229004    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-certs\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.229211 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.229184    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istiod-ca-cert\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.230887 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.230868    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-podinfo\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.230970 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.230905    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-envoy\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.241178 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.241151    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-token\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.241301 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.241258    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nclb\" (UniqueName: \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-kube-api-access-2nclb\") pod \"e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") " pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.333988 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.333959    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:51.379458 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.379421    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 03:59:51.638390 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.638352    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"]
Nov 04 03:59:51.639669 ip-10-0-129-253 kubenswrapper[6751]: W1104 03:59:51.639635    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e4dee6c_a29f_4b6b_a29b_19aa0799c8c1.slice/crio-fb51d8d22573054fcd2e5f8981a4fa6f70a45b2ee55241e1ca2b6d6a43f66f8d WatchSource:0}: Error finding container fb51d8d22573054fcd2e5f8981a4fa6f70a45b2ee55241e1ca2b6d6a43f66f8d: Status 404 returned error can't find the container with id fb51d8d22573054fcd2e5f8981a4fa6f70a45b2ee55241e1ca2b6d6a43f66f8d
Nov 04 03:59:51.642463 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.642423    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:51.642555 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.642516    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:51.642598 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.642558    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 03:59:51.888921 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:51.888852    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:52.122981 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.122947    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" event={"ID":"a174f8b4-9d90-407f-8914-107ddd97d3db","Type":"ContainerStarted","Data":"96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069"}
Nov 04 03:59:52.123175 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.123004    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" event={"ID":"a174f8b4-9d90-407f-8914-107ddd97d3db","Type":"ContainerStarted","Data":"529d33cdd59b6663023e0d71633314dace0a2ac825d933daf19eb38bcce73d9e"}
Nov 04 03:59:52.124917 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.124889    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" event={"ID":"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1","Type":"ContainerStarted","Data":"ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1"}
Nov 04 03:59:52.125044 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.124932    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" event={"ID":"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1","Type":"ContainerStarted","Data":"fb51d8d22573054fcd2e5f8981a4fa6f70a45b2ee55241e1ca2b6d6a43f66f8d"}
Nov 04 03:59:52.207700 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.200560    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" podStartSLOduration=2.20053769 podStartE2EDuration="2.20053769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:52.164601109 +0000 UTC m=+586.498538988" watchObservedRunningTime="2026-11-04 03:59:52.20053769 +0000 UTC m=+586.534475570"
Nov 04 03:59:52.334714 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.334688    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:52.336262 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.336235    6751 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 03:59:52.336390 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.336282    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 03:59:52.379597 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.379559    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:52.701088 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.701052    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:52.939447 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.939416    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:52.940147 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.940124    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 03:59:52.940206 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:52.940178    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" podUID="a174f8b4-9d90-407f-8914-107ddd97d3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 03:59:53.335408 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:53.334949    6751 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 03:59:53.335408 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:53.335012    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 03:59:53.400242 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:53.400154    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:53.938932 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:53.938893    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 03:59:53.939128 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:53.938955    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" podUID="a174f8b4-9d90-407f-8914-107ddd97d3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 03:59:54.177647 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:54.176271    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:54.334997 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:54.334907    6751 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 03:59:54.334997 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:54.334974    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 03:59:54.941641 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:54.940704    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 03:59:54.941641 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:54.940769    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" podUID="a174f8b4-9d90-407f-8914-107ddd97d3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 03:59:55.034248 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:55.034214    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:55.335213 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:55.335119    6751 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 03:59:55.335213 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:55.335180    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 03:59:55.785646 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:55.775191    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:55.939880 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:55.939848    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 03:59:55.940041 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:55.939904    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" podUID="a174f8b4-9d90-407f-8914-107ddd97d3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 03:59:56.335339 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:56.335309    6751 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 03:59:56.335831 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:56.335800    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 03:59:56.492651 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:56.492572    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:56.938991 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:56.938955    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 03:59:56.939150 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:56.939012    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" podUID="a174f8b4-9d90-407f-8914-107ddd97d3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 03:59:57.200771 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:57.200523    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:57.334539 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:57.334501    6751 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 03:59:57.334715 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:57.334559    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 03:59:57.907867 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:57.907821    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:58.139113 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:58.139078    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:58.139762 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:58.139735    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:58.140715 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:58.140693    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 03:59:58.167705 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:58.167563    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" podStartSLOduration=8.167541722 podStartE2EDuration="8.167541722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 03:59:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 03:59:52.217972607 +0000 UTC m=+586.551910486" watchObservedRunningTime="2026-11-04 03:59:58.167541722 +0000 UTC m=+592.501479600"
Nov 04 03:59:58.401112 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:58.401077    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:58.401765 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:58.401741    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:58.403346 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:58.403318    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 03:59:58.624099 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:58.624069    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:58.636690 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:58.636660    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 03:59:58.782131 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:58.782094    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 03:59:59.245838 ip-10-0-129-253 kubenswrapper[6751]: I1104 03:59:59.245804    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 04:00:00.000067 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.000029    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:00.173706 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.173206    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"]
Nov 04 04:00:00.182504 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.179582    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"]
Nov 04 04:00:00.187640 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.179716    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"
Nov 04 04:00:00.197789 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.194019    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"
Nov 04 04:00:00.210677 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.202275    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 04:00:00.210677 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.202399    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 04:00:00.212652 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.211054    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"]
Nov 04 04:00:00.235148 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.228384    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr\" (UID: \"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"
Nov 04 04:00:00.235148 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.228508    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8pwn\" (UniqueName: \"kubernetes.io/projected/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-kube-api-access-h8pwn\") pod \"maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr\" (UID: \"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"
Nov 04 04:00:00.240303 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.238718    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"]
Nov 04 04:00:00.240303 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.239797    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"
Nov 04 04:00:00.243980 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.243956    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"]
Nov 04 04:00:00.246662 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.244851    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"]
Nov 04 04:00:00.246662 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.244917    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"]
Nov 04 04:00:00.261096 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.261025    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"
Nov 04 04:00:00.281262 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.276674    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"]
Nov 04 04:00:00.281262 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.280611    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"]
Nov 04 04:00:00.284161 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.284114    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2wnx7\""
Nov 04 04:00:00.289492 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.289469    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"]
Nov 04 04:00:00.289600 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.289579    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"
Nov 04 04:00:00.329513 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.329482    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8pwn\" (UniqueName: \"kubernetes.io/projected/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-kube-api-access-h8pwn\") pod \"maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr\" (UID: \"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"
Nov 04 04:00:00.329701 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.329614    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf88847-cb00-4229-96ed-23a6bd28b221-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9\" (UID: \"ddf88847-cb00-4229-96ed-23a6bd28b221\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"
Nov 04 04:00:00.329790 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.329772    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/929adf06-b6db-449c-b2d7-4371adb39b74-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw\" (UID: \"929adf06-b6db-449c-b2d7-4371adb39b74\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"
Nov 04 04:00:00.329909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.329885    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr\" (UID: \"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"
Nov 04 04:00:00.329967 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.329928    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-996z8\" (UniqueName: \"kubernetes.io/projected/ddf88847-cb00-4229-96ed-23a6bd28b221-kube-api-access-996z8\") pod \"maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9\" (UID: \"ddf88847-cb00-4229-96ed-23a6bd28b221\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"
Nov 04 04:00:00.330030 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.329972    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b54c76b1-1055-4307-99fa-e1b55e2402e5-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896080-tvbcz\" (UID: \"b54c76b1-1055-4307-99fa-e1b55e2402e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"
Nov 04 04:00:00.330030 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.330012    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x25cp\" (UniqueName: \"kubernetes.io/projected/929adf06-b6db-449c-b2d7-4371adb39b74-kube-api-access-x25cp\") pod \"maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw\" (UID: \"929adf06-b6db-449c-b2d7-4371adb39b74\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"
Nov 04 04:00:00.330139 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.330066    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49t2b\" (UniqueName: \"kubernetes.io/projected/b54c76b1-1055-4307-99fa-e1b55e2402e5-kube-api-access-49t2b\") pod \"maas-api-key-cleanup-29896080-tvbcz\" (UID: \"b54c76b1-1055-4307-99fa-e1b55e2402e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"
Nov 04 04:00:00.330498 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.330474    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr\" (UID: \"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"
Nov 04 04:00:00.366795 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.366770    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8pwn\" (UniqueName: \"kubernetes.io/projected/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-kube-api-access-h8pwn\") pod \"maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr\" (UID: \"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"
Nov 04 04:00:00.379678 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.379650    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 04:00:00.392765 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.392744    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 04:00:00.430887 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.430866    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-996z8\" (UniqueName: \"kubernetes.io/projected/ddf88847-cb00-4229-96ed-23a6bd28b221-kube-api-access-996z8\") pod \"maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9\" (UID: \"ddf88847-cb00-4229-96ed-23a6bd28b221\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"
Nov 04 04:00:00.430993 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.430904    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b54c76b1-1055-4307-99fa-e1b55e2402e5-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896080-tvbcz\" (UID: \"b54c76b1-1055-4307-99fa-e1b55e2402e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"
Nov 04 04:00:00.430993 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.430924    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25cp\" (UniqueName: \"kubernetes.io/projected/929adf06-b6db-449c-b2d7-4371adb39b74-kube-api-access-x25cp\") pod \"maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw\" (UID: \"929adf06-b6db-449c-b2d7-4371adb39b74\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"
Nov 04 04:00:00.430993 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.430951    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49t2b\" (UniqueName: \"kubernetes.io/projected/b54c76b1-1055-4307-99fa-e1b55e2402e5-kube-api-access-49t2b\") pod \"maas-api-key-cleanup-29896080-tvbcz\" (UID: \"b54c76b1-1055-4307-99fa-e1b55e2402e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"
Nov 04 04:00:00.431213 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.431189    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf88847-cb00-4229-96ed-23a6bd28b221-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9\" (UID: \"ddf88847-cb00-4229-96ed-23a6bd28b221\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"
Nov 04 04:00:00.431290 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.431274    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/929adf06-b6db-449c-b2d7-4371adb39b74-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw\" (UID: \"929adf06-b6db-449c-b2d7-4371adb39b74\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"
Nov 04 04:00:00.431345 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.431307    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trfp6\" (UniqueName: \"kubernetes.io/projected/26a58172-dd31-4e0c-bf70-cd734f4c2251-kube-api-access-trfp6\") pod \"maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd\" (UID: \"26a58172-dd31-4e0c-bf70-cd734f4c2251\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"
Nov 04 04:00:00.431408 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.431342    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26a58172-dd31-4e0c-bf70-cd734f4c2251-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd\" (UID: \"26a58172-dd31-4e0c-bf70-cd734f4c2251\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"
Nov 04 04:00:00.431609 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.431586    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b54c76b1-1055-4307-99fa-e1b55e2402e5-maas-api-service-ca\") pod \"maas-api-key-cleanup-29896080-tvbcz\" (UID: \"b54c76b1-1055-4307-99fa-e1b55e2402e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"
Nov 04 04:00:00.431820 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.431803    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf88847-cb00-4229-96ed-23a6bd28b221-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9\" (UID: \"ddf88847-cb00-4229-96ed-23a6bd28b221\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"
Nov 04 04:00:00.431911 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.431894    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/929adf06-b6db-449c-b2d7-4371adb39b74-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw\" (UID: \"929adf06-b6db-449c-b2d7-4371adb39b74\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"
Nov 04 04:00:00.445515 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.445492    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x25cp\" (UniqueName: \"kubernetes.io/projected/929adf06-b6db-449c-b2d7-4371adb39b74-kube-api-access-x25cp\") pod \"maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw\" (UID: \"929adf06-b6db-449c-b2d7-4371adb39b74\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"
Nov 04 04:00:00.445610 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.445536    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49t2b\" (UniqueName: \"kubernetes.io/projected/b54c76b1-1055-4307-99fa-e1b55e2402e5-kube-api-access-49t2b\") pod \"maas-api-key-cleanup-29896080-tvbcz\" (UID: \"b54c76b1-1055-4307-99fa-e1b55e2402e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"
Nov 04 04:00:00.445715 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.445695    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-996z8\" (UniqueName: \"kubernetes.io/projected/ddf88847-cb00-4229-96ed-23a6bd28b221-kube-api-access-996z8\") pod \"maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9\" (UID: \"ddf88847-cb00-4229-96ed-23a6bd28b221\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"
Nov 04 04:00:00.532468 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.532381    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trfp6\" (UniqueName: \"kubernetes.io/projected/26a58172-dd31-4e0c-bf70-cd734f4c2251-kube-api-access-trfp6\") pod \"maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd\" (UID: \"26a58172-dd31-4e0c-bf70-cd734f4c2251\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"
Nov 04 04:00:00.532468 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.532442    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26a58172-dd31-4e0c-bf70-cd734f4c2251-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd\" (UID: \"26a58172-dd31-4e0c-bf70-cd734f4c2251\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"
Nov 04 04:00:00.533087 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.533067    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26a58172-dd31-4e0c-bf70-cd734f4c2251-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd\" (UID: \"26a58172-dd31-4e0c-bf70-cd734f4c2251\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"
Nov 04 04:00:00.544779 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.544754    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trfp6\" (UniqueName: \"kubernetes.io/projected/26a58172-dd31-4e0c-bf70-cd734f4c2251-kube-api-access-trfp6\") pod \"maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd\" (UID: \"26a58172-dd31-4e0c-bf70-cd734f4c2251\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"
Nov 04 04:00:00.579086 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.579067    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"
Nov 04 04:00:00.581851 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.581833    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"
Nov 04 04:00:00.593991 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.593966    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"
Nov 04 04:00:00.601247 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.601225    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"
Nov 04 04:00:00.610447 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.610419    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"
Nov 04 04:00:00.889794 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.889759    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"]
Nov 04 04:00:00.896368 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.896340    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"]
Nov 04 04:00:00.896557 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:00.896508    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca746e37_9aca_4cbd_8ea8_60c80a75e0c4.slice/crio-95b6f0d86c5e0be7762cded4198f60b8400a76ab698da74ca7eb282dbdd4cde4 WatchSource:0}: Error finding container 95b6f0d86c5e0be7762cded4198f60b8400a76ab698da74ca7eb282dbdd4cde4: Status 404 returned error can't find the container with id 95b6f0d86c5e0be7762cded4198f60b8400a76ab698da74ca7eb282dbdd4cde4
Nov 04 04:00:00.897114 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:00.897092    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod929adf06_b6db_449c_b2d7_4371adb39b74.slice/crio-58c7b889223579f1cc498e3a343ae677e7b08b08dc502d6575fb99ee58b5168a WatchSource:0}: Error finding container 58c7b889223579f1cc498e3a343ae677e7b08b08dc502d6575fb99ee58b5168a: Status 404 returned error can't find the container with id 58c7b889223579f1cc498e3a343ae677e7b08b08dc502d6575fb99ee58b5168a
Nov 04 04:00:00.910837 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.910807    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:00.911604 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.911586    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"]
Nov 04 04:00:00.918000 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:00.917953    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf88847_cb00_4229_96ed_23a6bd28b221.slice/crio-c94ce97dda7b2eb6a29c4ae9344e975e58de243c2b2babc4d73cf8a852a85798 WatchSource:0}: Error finding container c94ce97dda7b2eb6a29c4ae9344e975e58de243c2b2babc4d73cf8a852a85798: Status 404 returned error can't find the container with id c94ce97dda7b2eb6a29c4ae9344e975e58de243c2b2babc4d73cf8a852a85798
Nov 04 04:00:00.919834 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.919733    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"]
Nov 04 04:00:00.922891 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:00.922866    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54c76b1_1055_4307_99fa_e1b55e2402e5.slice/crio-ef7a1ac6924bf7f10e98f3365ae19b5ed853ee028f17185e73e68e740008b23c WatchSource:0}: Error finding container ef7a1ac6924bf7f10e98f3365ae19b5ed853ee028f17185e73e68e740008b23c: Status 404 returned error can't find the container with id ef7a1ac6924bf7f10e98f3365ae19b5ed853ee028f17185e73e68e740008b23c
Nov 04 04:00:00.944019 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:00.943990    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"]
Nov 04 04:00:00.945645 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:00.945604    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a58172_dd31_4e0c_bf70_cd734f4c2251.slice/crio-cb74e8e1a68172e120fd3ec6071710ed44f55b6e66c724f32b6180e97159bf88 WatchSource:0}: Error finding container cb74e8e1a68172e120fd3ec6071710ed44f55b6e66c724f32b6180e97159bf88: Status 404 returned error can't find the container with id cb74e8e1a68172e120fd3ec6071710ed44f55b6e66c724f32b6180e97159bf88
Nov 04 04:00:01.266102 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:01.266016    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" event={"ID":"26a58172-dd31-4e0c-bf70-cd734f4c2251","Type":"ContainerStarted","Data":"cb74e8e1a68172e120fd3ec6071710ed44f55b6e66c724f32b6180e97159bf88"}
Nov 04 04:00:01.267237 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:01.267216    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" event={"ID":"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4","Type":"ContainerStarted","Data":"95b6f0d86c5e0be7762cded4198f60b8400a76ab698da74ca7eb282dbdd4cde4"}
Nov 04 04:00:01.268407 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:01.268383    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz" event={"ID":"b54c76b1-1055-4307-99fa-e1b55e2402e5","Type":"ContainerStarted","Data":"ef7a1ac6924bf7f10e98f3365ae19b5ed853ee028f17185e73e68e740008b23c"}
Nov 04 04:00:01.269495 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:01.269475    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9" event={"ID":"ddf88847-cb00-4229-96ed-23a6bd28b221","Type":"ContainerStarted","Data":"c94ce97dda7b2eb6a29c4ae9344e975e58de243c2b2babc4d73cf8a852a85798"}
Nov 04 04:00:01.270593 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:01.270573    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" event={"ID":"929adf06-b6db-449c-b2d7-4371adb39b74","Type":"ContainerStarted","Data":"58c7b889223579f1cc498e3a343ae677e7b08b08dc502d6575fb99ee58b5168a"}
Nov 04 04:00:01.282225 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:01.282205    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 04:00:01.810234 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:01.810203    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:02.493048 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:02.493017    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:03.202401 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:03.202036    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:04.002348 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:04.002310    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:04.806407 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:04.806375    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:05.711599 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:05.711486    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:06.303889 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.303863    6751 generic.go:356] "Generic (PLEG): container finished" podID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerID="88a504414e379463358bf8a979c6246006002004dd043d87366ac224162b2fb0" exitCode=60
Nov 04 04:00:06.304002 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.303962    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz" event={"ID":"b54c76b1-1055-4307-99fa-e1b55e2402e5","Type":"ContainerDied","Data":"88a504414e379463358bf8a979c6246006002004dd043d87366ac224162b2fb0"}
Nov 04 04:00:06.304207 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.304177    6751 scope.go:122] "RemoveContainer" containerID="88a504414e379463358bf8a979c6246006002004dd043d87366ac224162b2fb0"
Nov 04 04:00:06.305789 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.305706    6751 generic.go:356] "Generic (PLEG): container finished" podID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerID="11c09cc2cad772944104e5b9536490648fdfc88159708889cb45982627201358" exitCode=60
Nov 04 04:00:06.305789 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.305759    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9" event={"ID":"ddf88847-cb00-4229-96ed-23a6bd28b221","Type":"ContainerDied","Data":"11c09cc2cad772944104e5b9536490648fdfc88159708889cb45982627201358"}
Nov 04 04:00:06.306023 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.306008    6751 scope.go:122] "RemoveContainer" containerID="11c09cc2cad772944104e5b9536490648fdfc88159708889cb45982627201358"
Nov 04 04:00:06.307688 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.307661    6751 generic.go:356] "Generic (PLEG): container finished" podID="929adf06-b6db-449c-b2d7-4371adb39b74" containerID="90928a7b18cc66ec79df8fa0ea295ddf9aa37fafeec056bff18add5d7d103cbb" exitCode=60
Nov 04 04:00:06.307762 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.307735    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" event={"ID":"929adf06-b6db-449c-b2d7-4371adb39b74","Type":"ContainerDied","Data":"90928a7b18cc66ec79df8fa0ea295ddf9aa37fafeec056bff18add5d7d103cbb"}
Nov 04 04:00:06.308103 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.308084    6751 scope.go:122] "RemoveContainer" containerID="90928a7b18cc66ec79df8fa0ea295ddf9aa37fafeec056bff18add5d7d103cbb"
Nov 04 04:00:06.309527 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.309495    6751 generic.go:356] "Generic (PLEG): container finished" podID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerID="5ba7dfc2d0a8939e8d6bd9c3d3dc26b362a810fc1f0044c08dc78557bac9df7a" exitCode=60
Nov 04 04:00:06.309595 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.309555    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" event={"ID":"26a58172-dd31-4e0c-bf70-cd734f4c2251","Type":"ContainerDied","Data":"5ba7dfc2d0a8939e8d6bd9c3d3dc26b362a810fc1f0044c08dc78557bac9df7a"}
Nov 04 04:00:06.309822 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.309787    6751 scope.go:122] "RemoveContainer" containerID="5ba7dfc2d0a8939e8d6bd9c3d3dc26b362a810fc1f0044c08dc78557bac9df7a"
Nov 04 04:00:06.504093 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.504061    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:06.760318 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:06.760237    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:07.333657 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.332742    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" event={"ID":"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4","Type":"ContainerStarted","Data":"77a8699c99cc0d3d1dabd0af6ec5629095663e02709582efdd9065d951fa5116"}
Nov 04 04:00:07.353684 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.348493    6751 generic.go:356] "Generic (PLEG): container finished" podID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerID="06037f31e17ee18bfbfd99997595970ef771ffc4ec8f953f6636f40f68861fb8" exitCode=60
Nov 04 04:00:07.353684 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.348962    6751 scope.go:122] "RemoveContainer" containerID="06037f31e17ee18bfbfd99997595970ef771ffc4ec8f953f6636f40f68861fb8"
Nov 04 04:00:07.353684 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:07.349423    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29896080-tvbcz_odh-ai-gateway-infra(b54c76b1-1055-4307-99fa-e1b55e2402e5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5"
Nov 04 04:00:07.353684 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.348982    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz" event={"ID":"b54c76b1-1055-4307-99fa-e1b55e2402e5","Type":"ContainerDied","Data":"06037f31e17ee18bfbfd99997595970ef771ffc4ec8f953f6636f40f68861fb8"}
Nov 04 04:00:07.353684 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.349800    6751 scope.go:122] "RemoveContainer" containerID="88a504414e379463358bf8a979c6246006002004dd043d87366ac224162b2fb0"
Nov 04 04:00:07.353684 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.353367    6751 generic.go:356] "Generic (PLEG): container finished" podID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerID="90de501d64661c9de4c3d62665628a08943e7606aa148703f68b736171ef6d94" exitCode=60
Nov 04 04:00:07.353684 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.353443    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9" event={"ID":"ddf88847-cb00-4229-96ed-23a6bd28b221","Type":"ContainerDied","Data":"90de501d64661c9de4c3d62665628a08943e7606aa148703f68b736171ef6d94"}
Nov 04 04:00:07.354540 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.353842    6751 scope.go:122] "RemoveContainer" containerID="90de501d64661c9de4c3d62665628a08943e7606aa148703f68b736171ef6d94"
Nov 04 04:00:07.354540 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:07.354140    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9_odh-ai-gateway-infra(ddf88847-cb00-4229-96ed-23a6bd28b221)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221"
Nov 04 04:00:07.357653 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.356885    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" event={"ID":"929adf06-b6db-449c-b2d7-4371adb39b74","Type":"ContainerStarted","Data":"954d66c1329af1f948dc41ee1b9fd30d42d2790f30376c4ae178ecc93688a05f"}
Nov 04 04:00:07.359612 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.359586    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" event={"ID":"26a58172-dd31-4e0c-bf70-cd734f4c2251","Type":"ContainerStarted","Data":"788787be6bb11dfc4bf5cf28ce765853cdcbc6fcfef9b6fee37d44f93b26afe0"}
Nov 04 04:00:07.364446 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.364404    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" podStartSLOduration=1.5150616430000001 podStartE2EDuration="7.364391996s" totalImagesPullingTime="5.849330353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:07.361903259 +0000 UTC m=+601.695841141" watchObservedRunningTime="2026-11-04 04:00:07.364391996 +0000 UTC m=+601.698329880"
Nov 04 04:00:07.373670 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.373651    6751 scope.go:122] "RemoveContainer" containerID="11c09cc2cad772944104e5b9536490648fdfc88159708889cb45982627201358"
Nov 04 04:00:07.380727 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.380687    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" podStartSLOduration=3.124090714 podStartE2EDuration="7.38067454s" totalImagesPullingTime="4.256583826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:07.379905289 +0000 UTC m=+601.713843170" watchObservedRunningTime="2026-11-04 04:00:07.38067454 +0000 UTC m=+601.714612417"
Nov 04 04:00:07.383611 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.383584    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:07.441700 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:07.441603    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" podStartSLOduration=3.377400755 podStartE2EDuration="7.441590189s" totalImagesPullingTime="4.064189434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:07.440279127 +0000 UTC m=+601.774217006" watchObservedRunningTime="2026-11-04 04:00:07.441590189 +0000 UTC m=+601.775528066"
Nov 04 04:00:08.161597 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.161559    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:08.366787 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.366751    6751 generic.go:356] "Generic (PLEG): container finished" podID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerID="77a8699c99cc0d3d1dabd0af6ec5629095663e02709582efdd9065d951fa5116" exitCode=60
Nov 04 04:00:08.366940 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.366813    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" event={"ID":"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4","Type":"ContainerDied","Data":"77a8699c99cc0d3d1dabd0af6ec5629095663e02709582efdd9065d951fa5116"}
Nov 04 04:00:08.367126 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.367111    6751 scope.go:122] "RemoveContainer" containerID="77a8699c99cc0d3d1dabd0af6ec5629095663e02709582efdd9065d951fa5116"
Nov 04 04:00:08.368835 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.368816    6751 scope.go:122] "RemoveContainer" containerID="06037f31e17ee18bfbfd99997595970ef771ffc4ec8f953f6636f40f68861fb8"
Nov 04 04:00:08.369110 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:08.369079    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29896080-tvbcz_odh-ai-gateway-infra(b54c76b1-1055-4307-99fa-e1b55e2402e5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5"
Nov 04 04:00:08.370651 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.370632    6751 scope.go:122] "RemoveContainer" containerID="90de501d64661c9de4c3d62665628a08943e7606aa148703f68b736171ef6d94"
Nov 04 04:00:08.370909 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:08.370884    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9_odh-ai-gateway-infra(ddf88847-cb00-4229-96ed-23a6bd28b221)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221"
Nov 04 04:00:08.372320 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.372299    6751 generic.go:356] "Generic (PLEG): container finished" podID="929adf06-b6db-449c-b2d7-4371adb39b74" containerID="954d66c1329af1f948dc41ee1b9fd30d42d2790f30376c4ae178ecc93688a05f" exitCode=60
Nov 04 04:00:08.372413 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.372376    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" event={"ID":"929adf06-b6db-449c-b2d7-4371adb39b74","Type":"ContainerDied","Data":"954d66c1329af1f948dc41ee1b9fd30d42d2790f30376c4ae178ecc93688a05f"}
Nov 04 04:00:08.372470 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.372434    6751 scope.go:122] "RemoveContainer" containerID="90928a7b18cc66ec79df8fa0ea295ddf9aa37fafeec056bff18add5d7d103cbb"
Nov 04 04:00:08.372577 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.372562    6751 scope.go:122] "RemoveContainer" containerID="954d66c1329af1f948dc41ee1b9fd30d42d2790f30376c4ae178ecc93688a05f"
Nov 04 04:00:08.372831 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:08.372806    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw_odh-ai-gateway-infra(929adf06-b6db-449c-b2d7-4371adb39b74)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" podUID="929adf06-b6db-449c-b2d7-4371adb39b74"
Nov 04 04:00:08.378397 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.378373    6751 generic.go:356] "Generic (PLEG): container finished" podID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerID="788787be6bb11dfc4bf5cf28ce765853cdcbc6fcfef9b6fee37d44f93b26afe0" exitCode=60
Nov 04 04:00:08.378496 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.378420    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" event={"ID":"26a58172-dd31-4e0c-bf70-cd734f4c2251","Type":"ContainerDied","Data":"788787be6bb11dfc4bf5cf28ce765853cdcbc6fcfef9b6fee37d44f93b26afe0"}
Nov 04 04:00:08.378707 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.378692    6751 scope.go:122] "RemoveContainer" containerID="788787be6bb11dfc4bf5cf28ce765853cdcbc6fcfef9b6fee37d44f93b26afe0"
Nov 04 04:00:08.378876 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:08.378860    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd_odh-ai-gateway-infra(26a58172-dd31-4e0c-bf70-cd734f4c2251)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251"
Nov 04 04:00:08.399330 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.399312    6751 scope.go:122] "RemoveContainer" containerID="5ba7dfc2d0a8939e8d6bd9c3d3dc26b362a810fc1f0044c08dc78557bac9df7a"
Nov 04 04:00:08.989742 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:08.989648    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:09.386568 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:09.386538    6751 scope.go:122] "RemoveContainer" containerID="954d66c1329af1f948dc41ee1b9fd30d42d2790f30376c4ae178ecc93688a05f"
Nov 04 04:00:09.387013 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:09.386871    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw_odh-ai-gateway-infra(929adf06-b6db-449c-b2d7-4371adb39b74)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" podUID="929adf06-b6db-449c-b2d7-4371adb39b74"
Nov 04 04:00:09.388252 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:09.388230    6751 scope.go:122] "RemoveContainer" containerID="788787be6bb11dfc4bf5cf28ce765853cdcbc6fcfef9b6fee37d44f93b26afe0"
Nov 04 04:00:09.388488 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:09.388468    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd_odh-ai-gateway-infra(26a58172-dd31-4e0c-bf70-cd734f4c2251)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251"
Nov 04 04:00:09.389728 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:09.389706    6751 generic.go:356] "Generic (PLEG): container finished" podID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerID="6827e60b1b7a59772dc1cca0c51ed6ae94c2ad0b9d65bb3e870c247bfa8580ab" exitCode=60
Nov 04 04:00:09.389819 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:09.389786    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" event={"ID":"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4","Type":"ContainerDied","Data":"6827e60b1b7a59772dc1cca0c51ed6ae94c2ad0b9d65bb3e870c247bfa8580ab"}
Nov 04 04:00:09.389864 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:09.389845    6751 scope.go:122] "RemoveContainer" containerID="77a8699c99cc0d3d1dabd0af6ec5629095663e02709582efdd9065d951fa5116"
Nov 04 04:00:09.390016 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:09.389988    6751 scope.go:122] "RemoveContainer" containerID="6827e60b1b7a59772dc1cca0c51ed6ae94c2ad0b9d65bb3e870c247bfa8580ab"
Nov 04 04:00:09.390270 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:09.390249    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr_odh-ai-gateway-infra(ca746e37-9aca-4cbd-8ea8-60c80a75e0c4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4"
Nov 04 04:00:09.987367 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:09.987333    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:10.397405 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:10.397373    6751 scope.go:122] "RemoveContainer" containerID="6827e60b1b7a59772dc1cca0c51ed6ae94c2ad0b9d65bb3e870c247bfa8580ab"
Nov 04 04:00:10.397809 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:10.397642    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr_odh-ai-gateway-infra(ca746e37-9aca-4cbd-8ea8-60c80a75e0c4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4"
Nov 04 04:00:10.905631 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:10.905597    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:11.608745 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:11.608717    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:12.514816 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:12.514770    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:13.310661 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:13.307062    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:14.123803 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:14.123772    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:15.097494 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:15.097459    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:15.440418 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:15.440326    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:15.730365 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:15.730285    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:16.207693 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:16.207649    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:17.119416 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:17.119381    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:18.004921 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:18.004887    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:18.779015 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:18.778979    6751 scope.go:122] "RemoveContainer" containerID="06037f31e17ee18bfbfd99997595970ef771ffc4ec8f953f6636f40f68861fb8"
Nov 04 04:00:18.779233 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:18.779215    6751 scope.go:122] "RemoveContainer" containerID="788787be6bb11dfc4bf5cf28ce765853cdcbc6fcfef9b6fee37d44f93b26afe0"
Nov 04 04:00:18.892569 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:18.892538    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:19.463292 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:19.463260    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" event={"ID":"26a58172-dd31-4e0c-bf70-cd734f4c2251","Type":"ContainerStarted","Data":"9585ff587003a44c657c2131e242d2d03c9ccec6a250d787dd9b9bf09cda0788"}
Nov 04 04:00:19.465106 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:19.465082    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz" event={"ID":"b54c76b1-1055-4307-99fa-e1b55e2402e5","Type":"ContainerStarted","Data":"5ad4de72729625ce1127ac4fc412ca81a96b37b3d14c4555dc9b833de8b59962"}
Nov 04 04:00:19.503488 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:19.503404    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz" podStartSLOduration=15.267846312 podStartE2EDuration="19.503380719s" totalImagesPullingTime="4.235534407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:19.500004719 +0000 UTC m=+613.833942601" watchObservedRunningTime="2026-11-04 04:00:19.503380719 +0000 UTC m=+613.837318599"
Nov 04 04:00:19.778648 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:19.778570    6751 scope.go:122] "RemoveContainer" containerID="90de501d64661c9de4c3d62665628a08943e7606aa148703f68b736171ef6d94"
Nov 04 04:00:19.884317 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:19.884280    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:20.472793 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.472763    6751 generic.go:356] "Generic (PLEG): container finished" podID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerID="9585ff587003a44c657c2131e242d2d03c9ccec6a250d787dd9b9bf09cda0788" exitCode=60
Nov 04 04:00:20.473244 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.472834    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" event={"ID":"26a58172-dd31-4e0c-bf70-cd734f4c2251","Type":"ContainerDied","Data":"9585ff587003a44c657c2131e242d2d03c9ccec6a250d787dd9b9bf09cda0788"}
Nov 04 04:00:20.473244 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.472896    6751 scope.go:122] "RemoveContainer" containerID="788787be6bb11dfc4bf5cf28ce765853cdcbc6fcfef9b6fee37d44f93b26afe0"
Nov 04 04:00:20.473244 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.473203    6751 scope.go:122] "RemoveContainer" containerID="9585ff587003a44c657c2131e242d2d03c9ccec6a250d787dd9b9bf09cda0788"
Nov 04 04:00:20.473438 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:20.473418    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd_odh-ai-gateway-infra(26a58172-dd31-4e0c-bf70-cd734f4c2251)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251"
Nov 04 04:00:20.475278 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.475247    6751 generic.go:356] "Generic (PLEG): container finished" podID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerID="5ad4de72729625ce1127ac4fc412ca81a96b37b3d14c4555dc9b833de8b59962" exitCode=60
Nov 04 04:00:20.475406 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.475316    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz" event={"ID":"b54c76b1-1055-4307-99fa-e1b55e2402e5","Type":"ContainerDied","Data":"5ad4de72729625ce1127ac4fc412ca81a96b37b3d14c4555dc9b833de8b59962"}
Nov 04 04:00:20.475692 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.475675    6751 scope.go:122] "RemoveContainer" containerID="5ad4de72729625ce1127ac4fc412ca81a96b37b3d14c4555dc9b833de8b59962"
Nov 04 04:00:20.475955 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:20.475931    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29896080-tvbcz_odh-ai-gateway-infra(b54c76b1-1055-4307-99fa-e1b55e2402e5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5"
Nov 04 04:00:20.477187 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.477163    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9" event={"ID":"ddf88847-cb00-4229-96ed-23a6bd28b221","Type":"ContainerStarted","Data":"aad7624148455167d9f896052b1debbd38fb5f8dbbcc68df776b83369071ecda"}
Nov 04 04:00:20.484553 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.484537    6751 scope.go:122] "RemoveContainer" containerID="06037f31e17ee18bfbfd99997595970ef771ffc4ec8f953f6636f40f68861fb8"
Nov 04 04:00:20.494880 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.494860    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"]
Nov 04 04:00:20.514260 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.514240    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"]
Nov 04 04:00:20.532301 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.532252    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9" podStartSLOduration=16.098790475 podStartE2EDuration="20.532237185s" totalImagesPullingTime="4.43344671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:20.529699111 +0000 UTC m=+614.863636992" watchObservedRunningTime="2026-11-04 04:00:20.532237185 +0000 UTC m=+614.866175064"
Nov 04 04:00:20.779251 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:20.779219    6751 scope.go:122] "RemoveContainer" containerID="954d66c1329af1f948dc41ee1b9fd30d42d2790f30376c4ae178ecc93688a05f"
Nov 04 04:00:21.085782 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:21.085754    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 04:00:21.486688 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.486662    6751 generic.go:356] "Generic (PLEG): container finished" podID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerID="aad7624148455167d9f896052b1debbd38fb5f8dbbcc68df776b83369071ecda" exitCode=60
Nov 04 04:00:21.487038 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.486732    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9" event={"ID":"ddf88847-cb00-4229-96ed-23a6bd28b221","Type":"ContainerDied","Data":"aad7624148455167d9f896052b1debbd38fb5f8dbbcc68df776b83369071ecda"}
Nov 04 04:00:21.487038 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.486798    6751 scope.go:122] "RemoveContainer" containerID="90de501d64661c9de4c3d62665628a08943e7606aa148703f68b736171ef6d94"
Nov 04 04:00:21.487153 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.487134    6751 scope.go:122] "RemoveContainer" containerID="aad7624148455167d9f896052b1debbd38fb5f8dbbcc68df776b83369071ecda"
Nov 04 04:00:21.487440 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:21.487412    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9_odh-ai-gateway-infra(ddf88847-cb00-4229-96ed-23a6bd28b221)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221"
Nov 04 04:00:21.488908 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.488889    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" event={"ID":"929adf06-b6db-449c-b2d7-4371adb39b74","Type":"ContainerStarted","Data":"8eec38ce6c9e31ef3838b52fdae1f4ffd7708b050f516fd87cbf7058990d3f06"}
Nov 04 04:00:21.544067 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.544034    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"]
Nov 04 04:00:21.610529 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.609365    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:21.658181 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.658147    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"
Nov 04 04:00:21.745292 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.745228    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"
Nov 04 04:00:21.771251 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.771222    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b54c76b1-1055-4307-99fa-e1b55e2402e5-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b54c76b1-1055-4307-99fa-e1b55e2402e5-maas-api-service-ca\") pod \"b54c76b1-1055-4307-99fa-e1b55e2402e5\" (UID: \"b54c76b1-1055-4307-99fa-e1b55e2402e5\") "
Nov 04 04:00:21.771893 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.771850    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b54c76b1-1055-4307-99fa-e1b55e2402e5-maas-api-service-ca" pod "b54c76b1-1055-4307-99fa-e1b55e2402e5" (UID: "b54c76b1-1055-4307-99fa-e1b55e2402e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:00:21.771995 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.771911    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b54c76b1-1055-4307-99fa-e1b55e2402e5-kube-api-access-49t2b\" (UniqueName: \"kubernetes.io/projected/b54c76b1-1055-4307-99fa-e1b55e2402e5-kube-api-access-49t2b\") pod \"b54c76b1-1055-4307-99fa-e1b55e2402e5\" (UID: \"b54c76b1-1055-4307-99fa-e1b55e2402e5\") "
Nov 04 04:00:21.772414 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.772390    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b54c76b1-1055-4307-99fa-e1b55e2402e5-maas-api-service-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:21.774036 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.774014    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b54c76b1-1055-4307-99fa-e1b55e2402e5-kube-api-access-49t2b" pod "b54c76b1-1055-4307-99fa-e1b55e2402e5" (UID: "b54c76b1-1055-4307-99fa-e1b55e2402e5"). InnerVolumeSpecName "kube-api-access-49t2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:00:21.778688 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.778666    6751 scope.go:122] "RemoveContainer" containerID="6827e60b1b7a59772dc1cca0c51ed6ae94c2ad0b9d65bb3e870c247bfa8580ab"
Nov 04 04:00:21.814525 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.814500    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:21.873211 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.873188    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/26a58172-dd31-4e0c-bf70-cd734f4c2251-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26a58172-dd31-4e0c-bf70-cd734f4c2251-maas-api-service-ca\") pod \"26a58172-dd31-4e0c-bf70-cd734f4c2251\" (UID: \"26a58172-dd31-4e0c-bf70-cd734f4c2251\") "
Nov 04 04:00:21.873311 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.873256    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26a58172-dd31-4e0c-bf70-cd734f4c2251-kube-api-access-trfp6\" (UniqueName: \"kubernetes.io/projected/26a58172-dd31-4e0c-bf70-cd734f4c2251-kube-api-access-trfp6\") pod \"26a58172-dd31-4e0c-bf70-cd734f4c2251\" (UID: \"26a58172-dd31-4e0c-bf70-cd734f4c2251\") "
Nov 04 04:00:21.873572 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.873550    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26a58172-dd31-4e0c-bf70-cd734f4c2251-maas-api-service-ca" pod "26a58172-dd31-4e0c-bf70-cd734f4c2251" (UID: "26a58172-dd31-4e0c-bf70-cd734f4c2251"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:00:21.873966 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.873944    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49t2b\" (UniqueName: \"kubernetes.io/projected/b54c76b1-1055-4307-99fa-e1b55e2402e5-kube-api-access-49t2b\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:21.874062 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.873971    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26a58172-dd31-4e0c-bf70-cd734f4c2251-maas-api-service-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:21.875318 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.875291    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a58172-dd31-4e0c-bf70-cd734f4c2251-kube-api-access-trfp6" pod "26a58172-dd31-4e0c-bf70-cd734f4c2251" (UID: "26a58172-dd31-4e0c-bf70-cd734f4c2251"). InnerVolumeSpecName "kube-api-access-trfp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:00:21.976842 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:21.976804    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trfp6\" (UniqueName: \"kubernetes.io/projected/26a58172-dd31-4e0c-bf70-cd734f4c2251-kube-api-access-trfp6\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:22.019881 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.019809    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:22.499860 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.499817    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd" event={"ID":"26a58172-dd31-4e0c-bf70-cd734f4c2251","Type":"ContainerDied","Data":"cb74e8e1a68172e120fd3ec6071710ed44f55b6e66c724f32b6180e97159bf88"}
Nov 04 04:00:22.500323 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.499938    6751 scope.go:122] "RemoveContainer" containerID="9585ff587003a44c657c2131e242d2d03c9ccec6a250d787dd9b9bf09cda0788"
Nov 04 04:00:22.500546 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.500516    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"
Nov 04 04:00:22.503404 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.503376    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" event={"ID":"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4","Type":"ContainerStarted","Data":"3075f91d089dc4a50bda4cd83f437257ae130f98703432a74bcf406775417c81"}
Nov 04 04:00:22.505238 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.505208    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz" event={"ID":"b54c76b1-1055-4307-99fa-e1b55e2402e5","Type":"ContainerDied","Data":"ef7a1ac6924bf7f10e98f3365ae19b5ed853ee028f17185e73e68e740008b23c"}
Nov 04 04:00:22.505709 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.505686    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"
Nov 04 04:00:22.511456 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.511433    6751 generic.go:356] "Generic (PLEG): container finished" podID="929adf06-b6db-449c-b2d7-4371adb39b74" containerID="8eec38ce6c9e31ef3838b52fdae1f4ffd7708b050f516fd87cbf7058990d3f06" exitCode=60
Nov 04 04:00:22.511572 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.511477    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" event={"ID":"929adf06-b6db-449c-b2d7-4371adb39b74","Type":"ContainerDied","Data":"8eec38ce6c9e31ef3838b52fdae1f4ffd7708b050f516fd87cbf7058990d3f06"}
Nov 04 04:00:22.511789 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.511772    6751 scope.go:122] "RemoveContainer" containerID="8eec38ce6c9e31ef3838b52fdae1f4ffd7708b050f516fd87cbf7058990d3f06"
Nov 04 04:00:22.512125 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:22.512059    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw_odh-ai-gateway-infra(929adf06-b6db-449c-b2d7-4371adb39b74)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" podUID="929adf06-b6db-449c-b2d7-4371adb39b74"
Nov 04 04:00:22.512125 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.512097    6751 scope.go:122] "RemoveContainer" containerID="5ad4de72729625ce1127ac4fc412ca81a96b37b3d14c4555dc9b833de8b59962"
Nov 04 04:00:22.520076 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.520055    6751 scope.go:122] "RemoveContainer" containerID="954d66c1329af1f948dc41ee1b9fd30d42d2790f30376c4ae178ecc93688a05f"
Nov 04 04:00:22.577775 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.577533    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"]
Nov 04 04:00:22.594873 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.594847    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"]
Nov 04 04:00:22.598755 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.598732    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29896080-tvbcz"]
Nov 04 04:00:22.612685 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.612661    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"]
Nov 04 04:00:22.616987 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.616962    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-56a44f-29896080-qlzjd"]
Nov 04 04:00:22.644094 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.644076    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"
Nov 04 04:00:22.785985 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.785897    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddf88847-cb00-4229-96ed-23a6bd28b221-kube-api-access-996z8\" (UniqueName: \"kubernetes.io/projected/ddf88847-cb00-4229-96ed-23a6bd28b221-kube-api-access-996z8\") pod \"ddf88847-cb00-4229-96ed-23a6bd28b221\" (UID: \"ddf88847-cb00-4229-96ed-23a6bd28b221\") "
Nov 04 04:00:22.785985 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.785959    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ddf88847-cb00-4229-96ed-23a6bd28b221-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf88847-cb00-4229-96ed-23a6bd28b221-maas-api-service-ca\") pod \"ddf88847-cb00-4229-96ed-23a6bd28b221\" (UID: \"ddf88847-cb00-4229-96ed-23a6bd28b221\") "
Nov 04 04:00:22.786346 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.786315    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf88847-cb00-4229-96ed-23a6bd28b221-maas-api-service-ca" pod "ddf88847-cb00-4229-96ed-23a6bd28b221" (UID: "ddf88847-cb00-4229-96ed-23a6bd28b221"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:00:22.786536 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.786516    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf88847-cb00-4229-96ed-23a6bd28b221-maas-api-service-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:22.788072 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.788051    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf88847-cb00-4229-96ed-23a6bd28b221-kube-api-access-996z8" pod "ddf88847-cb00-4229-96ed-23a6bd28b221" (UID: "ddf88847-cb00-4229-96ed-23a6bd28b221"). InnerVolumeSpecName "kube-api-access-996z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:00:22.798853 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.798826    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:22.887651 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:22.887611    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-996z8\" (UniqueName: \"kubernetes.io/projected/ddf88847-cb00-4229-96ed-23a6bd28b221-kube-api-access-996z8\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:23.501028 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.500994    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:23.519420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.519395    6751 generic.go:356] "Generic (PLEG): container finished" podID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerID="3075f91d089dc4a50bda4cd83f437257ae130f98703432a74bcf406775417c81" exitCode=60
Nov 04 04:00:23.519552 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.519469    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" event={"ID":"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4","Type":"ContainerDied","Data":"3075f91d089dc4a50bda4cd83f437257ae130f98703432a74bcf406775417c81"}
Nov 04 04:00:23.519552 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.519527    6751 scope.go:122] "RemoveContainer" containerID="6827e60b1b7a59772dc1cca0c51ed6ae94c2ad0b9d65bb3e870c247bfa8580ab"
Nov 04 04:00:23.519946 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.519925    6751 scope.go:122] "RemoveContainer" containerID="3075f91d089dc4a50bda4cd83f437257ae130f98703432a74bcf406775417c81"
Nov 04 04:00:23.520269 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:23.520250    6751 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr_odh-ai-gateway-infra(ca746e37-9aca-4cbd-8ea8-60c80a75e0c4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4"
Nov 04 04:00:23.522053 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.522035    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"
Nov 04 04:00:23.522157 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.522055    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9" event={"ID":"ddf88847-cb00-4229-96ed-23a6bd28b221","Type":"ContainerDied","Data":"c94ce97dda7b2eb6a29c4ae9344e975e58de243c2b2babc4d73cf8a852a85798"}
Nov 04 04:00:23.532257 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.532236    6751 scope.go:122] "RemoveContainer" containerID="aad7624148455167d9f896052b1debbd38fb5f8dbbcc68df776b83369071ecda"
Nov 04 04:00:23.548283 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.548249    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"]
Nov 04 04:00:23.583410 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.583383    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"]
Nov 04 04:00:23.587140 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.587113    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-6a9cdb-29896080-rbdm9"]
Nov 04 04:00:23.688788 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.688767    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"
Nov 04 04:00:23.784818 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.784743    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251" path="/var/lib/kubelet/pods/26a58172-dd31-4e0c-bf70-cd734f4c2251/volumes"
Nov 04 04:00:23.785226 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.785192    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5" path="/var/lib/kubelet/pods/b54c76b1-1055-4307-99fa-e1b55e2402e5/volumes"
Nov 04 04:00:23.785648 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.785600    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221" path="/var/lib/kubelet/pods/ddf88847-cb00-4229-96ed-23a6bd28b221/volumes"
Nov 04 04:00:23.797137 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.797116    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/929adf06-b6db-449c-b2d7-4371adb39b74-kube-api-access-x25cp\" (UniqueName: \"kubernetes.io/projected/929adf06-b6db-449c-b2d7-4371adb39b74-kube-api-access-x25cp\") pod \"929adf06-b6db-449c-b2d7-4371adb39b74\" (UID: \"929adf06-b6db-449c-b2d7-4371adb39b74\") "
Nov 04 04:00:23.797226 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.797172    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/929adf06-b6db-449c-b2d7-4371adb39b74-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/929adf06-b6db-449c-b2d7-4371adb39b74-maas-api-service-ca\") pod \"929adf06-b6db-449c-b2d7-4371adb39b74\" (UID: \"929adf06-b6db-449c-b2d7-4371adb39b74\") "
Nov 04 04:00:23.797592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.797571    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/929adf06-b6db-449c-b2d7-4371adb39b74-maas-api-service-ca" pod "929adf06-b6db-449c-b2d7-4371adb39b74" (UID: "929adf06-b6db-449c-b2d7-4371adb39b74"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:00:23.799034 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.799012    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/929adf06-b6db-449c-b2d7-4371adb39b74-kube-api-access-x25cp" pod "929adf06-b6db-449c-b2d7-4371adb39b74" (UID: "929adf06-b6db-449c-b2d7-4371adb39b74"). InnerVolumeSpecName "kube-api-access-x25cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:00:23.899272 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.899116    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x25cp\" (UniqueName: \"kubernetes.io/projected/929adf06-b6db-449c-b2d7-4371adb39b74-kube-api-access-x25cp\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:23.899272 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:23.899146    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/929adf06-b6db-449c-b2d7-4371adb39b74-maas-api-service-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:24.296490 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.296456    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:24.533183 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.533155    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw" event={"ID":"929adf06-b6db-449c-b2d7-4371adb39b74","Type":"ContainerDied","Data":"58c7b889223579f1cc498e3a343ae677e7b08b08dc502d6575fb99ee58b5168a"}
Nov 04 04:00:24.533546 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.533240    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"
Nov 04 04:00:24.533546 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.533253    6751 scope.go:122] "RemoveContainer" containerID="8eec38ce6c9e31ef3838b52fdae1f4ffd7708b050f516fd87cbf7058990d3f06"
Nov 04 04:00:24.566266 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.566150    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"]
Nov 04 04:00:24.570272 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.570245    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-78ca37-29896080-tdsqw"]
Nov 04 04:00:24.676588 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.676564    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"
Nov 04 04:00:24.809753 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.809724    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-kube-api-access-h8pwn\" (UniqueName: \"kubernetes.io/projected/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-kube-api-access-h8pwn\") pod \"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4\" (UID: \"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4\") "
Nov 04 04:00:24.809899 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.809886    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-maas-api-service-ca\") pod \"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4\" (UID: \"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4\") "
Nov 04 04:00:24.810248 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.810220    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-maas-api-service-ca" pod "ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" (UID: "ca746e37-9aca-4cbd-8ea8-60c80a75e0c4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:00:24.811716 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.811694    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-kube-api-access-h8pwn" pod "ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" (UID: "ca746e37-9aca-4cbd-8ea8-60c80a75e0c4"). InnerVolumeSpecName "kube-api-access-h8pwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:00:24.911566 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.911540    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8pwn\" (UniqueName: \"kubernetes.io/projected/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-kube-api-access-h8pwn\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:24.911687 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:24.911569    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4-maas-api-service-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:25.000837 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:25.000785    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:25.545156 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:25.545130    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"
Nov 04 04:00:25.545667 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:25.545157    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr" event={"ID":"ca746e37-9aca-4cbd-8ea8-60c80a75e0c4","Type":"ContainerDied","Data":"95b6f0d86c5e0be7762cded4198f60b8400a76ab698da74ca7eb282dbdd4cde4"}
Nov 04 04:00:25.545667 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:25.545217    6751 scope.go:122] "RemoveContainer" containerID="3075f91d089dc4a50bda4cd83f437257ae130f98703432a74bcf406775417c81"
Nov 04 04:00:25.574005 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:25.573979    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"]
Nov 04 04:00:25.588510 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:25.588490    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4afe0e-29896080-vhhwr"]
Nov 04 04:00:25.786139 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:25.786094    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="929adf06-b6db-449c-b2d7-4371adb39b74" path="/var/lib/kubelet/pods/929adf06-b6db-449c-b2d7-4371adb39b74/volumes"
Nov 04 04:00:25.786797 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:25.786776    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" path="/var/lib/kubelet/pods/ca746e37-9aca-4cbd-8ea8-60c80a75e0c4/volumes"
Nov 04 04:00:25.910403 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:25.910367    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:26.864197 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:26.864168    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:27.885854 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:27.885812    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:29.003298 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:29.003252    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:29.819913 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:29.819872    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:30.653182 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:30.653148    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:31.497812 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:31.497779    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:32.252386 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:32.252353    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:33.217079 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:33.217040    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:33.988799 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:33.988754    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:34.708202 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:34.708163    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:35.520586 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:35.520547    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:36.377742 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:36.377713    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:37.115992 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:37.115953    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:37.958460 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:37.958424    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:38.737012 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:38.736976    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:39.728078 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:39.728046    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:39.938229 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:39.938185    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:41.007940 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:41.007902    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:41.817030 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:41.816982    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:42.684866 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.684828    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:42.864088 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864056    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6866-d6274f"]
Nov 04 04:00:42.864487 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864475    6751 cpu_manager.go:414] "containerMap: removing container" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerName="cleanup"
Nov 04 04:00:42.864539 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864489    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerName="cleanup"
Nov 04 04:00:42.864539 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864500    6751 cpu_manager.go:414] "containerMap: removing container" podUID="929adf06-b6db-449c-b2d7-4371adb39b74" containerName="cleanup"
Nov 04 04:00:42.864539 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864505    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="929adf06-b6db-449c-b2d7-4371adb39b74" containerName="cleanup"
Nov 04 04:00:42.864539 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864514    6751 cpu_manager.go:414] "containerMap: removing container" podUID="929adf06-b6db-449c-b2d7-4371adb39b74" containerName="cleanup"
Nov 04 04:00:42.864539 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864519    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="929adf06-b6db-449c-b2d7-4371adb39b74" containerName="cleanup"
Nov 04 04:00:42.864539 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864524    6751 cpu_manager.go:414] "containerMap: removing container" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerName="cleanup"
Nov 04 04:00:42.864539 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864530    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864552    6751 cpu_manager.go:414] "containerMap: removing container" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864556    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864562    6751 cpu_manager.go:414] "containerMap: removing container" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864567    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864574    6751 cpu_manager.go:414] "containerMap: removing container" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864578    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864583    6751 cpu_manager.go:414] "containerMap: removing container" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864587    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864592    6751 cpu_manager.go:414] "containerMap: removing container" podUID="929adf06-b6db-449c-b2d7-4371adb39b74" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864596    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="929adf06-b6db-449c-b2d7-4371adb39b74" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864604    6751 cpu_manager.go:414] "containerMap: removing container" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864608    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864615    6751 cpu_manager.go:414] "containerMap: removing container" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864633    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864640    6751 cpu_manager.go:414] "containerMap: removing container" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864644    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864653    6751 cpu_manager.go:414] "containerMap: removing container" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864657    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864667    6751 cpu_manager.go:414] "containerMap: removing container" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864671    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864677    6751 cpu_manager.go:414] "containerMap: removing container" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864681    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864743    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="929adf06-b6db-449c-b2d7-4371adb39b74" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864750    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="929adf06-b6db-449c-b2d7-4371adb39b74" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864755    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864764    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864769    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864774    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864780    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864785    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="929adf06-b6db-449c-b2d7-4371adb39b74" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864791    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864795    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca746e37-9aca-4cbd-8ea8-60c80a75e0c4" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864800    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerName="cleanup"
Nov 04 04:00:42.864792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864806    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="26a58172-dd31-4e0c-bf70-cd734f4c2251" containerName="cleanup"
Nov 04 04:00:42.865730 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864811    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerName="cleanup"
Nov 04 04:00:42.865730 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.864818    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="b54c76b1-1055-4307-99fa-e1b55e2402e5" containerName="cleanup"
Nov 04 04:00:42.869351 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.869334    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6866-d6274f"
Nov 04 04:00:42.874583 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.874560    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6866-d6274f"]
Nov 04 04:00:42.934090 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:42.934040    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zg68j\""
Nov 04 04:00:43.015483 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:43.015425    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgfnl\" (UniqueName: \"kubernetes.io/projected/f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9-kube-api-access-vgfnl\") pod \"test-curl-6866-d6274f\" (UID: \"f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9\") " pod="opendatahub/test-curl-6866-d6274f"
Nov 04 04:00:43.116201 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:43.116156    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgfnl\" (UniqueName: \"kubernetes.io/projected/f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9-kube-api-access-vgfnl\") pod \"test-curl-6866-d6274f\" (UID: \"f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9\") " pod="opendatahub/test-curl-6866-d6274f"
Nov 04 04:00:43.128672 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:43.128644    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgfnl\" (UniqueName: \"kubernetes.io/projected/f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9-kube-api-access-vgfnl\") pod \"test-curl-6866-d6274f\" (UID: \"f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9\") " pod="opendatahub/test-curl-6866-d6274f"
Nov 04 04:00:43.183606 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:43.183580    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6866-d6274f"
Nov 04 04:00:43.488918 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:43.488828    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6866-d6274f"]
Nov 04 04:00:43.491718 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:43.491678    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf98d9cbe_fe5a_4982_9c8e_b8371b2b76f9.slice/crio-a1e145931d6de3512a95637ed366fca1d0596eb93735b38220ba45b34248a400 WatchSource:0}: Error finding container a1e145931d6de3512a95637ed366fca1d0596eb93735b38220ba45b34248a400: Status 404 returned error can't find the container with id a1e145931d6de3512a95637ed366fca1d0596eb93735b38220ba45b34248a400
Nov 04 04:00:43.498919 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:43.498895    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:43.665129 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:43.665100    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6866-d6274f" event={"ID":"f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9","Type":"ContainerStarted","Data":"a1e145931d6de3512a95637ed366fca1d0596eb93735b38220ba45b34248a400"}
Nov 04 04:00:44.292697 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:44.292668    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:45.103002 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:45.102962    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:45.802288 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:45.802230    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:46.520604 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:46.520567    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:46.684197 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:46.684170    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6866-d6274f" event={"ID":"f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9","Type":"ContainerStarted","Data":"c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc"}
Nov 04 04:00:46.704223 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:46.704175    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6866-d6274f" podStartSLOduration=1.882690859 podStartE2EDuration="4.704162319s" totalImagesPullingTime="2.82147146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:46.703236987 +0000 UTC m=+641.037174863" watchObservedRunningTime="2026-11-04 04:00:46.704162319 +0000 UTC m=+641.038100197"
Nov 04 04:00:47.357241 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:47.357201    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:47.415741 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:47.415698    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6866-d6274f"]
Nov 04 04:00:47.419297 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:47.419271    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6866-d6274f"]
Nov 04 04:00:48.281643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.264209    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:48.520267 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.520237    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"]
Nov 04 04:00:48.520705 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.520691    6751 cpu_manager.go:414] "containerMap: removing container" podUID="f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9" containerName="test-curl-6866-d6274f"
Nov 04 04:00:48.520768 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.520706    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9" containerName="test-curl-6866-d6274f"
Nov 04 04:00:48.520804 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.520786    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9" containerName="test-curl-6866-d6274f"
Nov 04 04:00:48.520804 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.520793    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddf88847-cb00-4229-96ed-23a6bd28b221" containerName="cleanup"
Nov 04 04:00:48.524126 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.524110    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:48.545952 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.545901    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 04:00:48.547150 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.547131    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2wnx7\""
Nov 04 04:00:48.548317 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.548295    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 04:00:48.555346 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.555320    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:48.567037 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.567017    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"]
Nov 04 04:00:48.572039 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.572017    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 04:00:48.606071 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.606042    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xws97\" (UniqueName: \"kubernetes.io/projected/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-kube-api-access-xws97\") pod \"maas-api-revoke-sub-914d691e-sxlhv\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:48.606213 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.606178    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-914d691e-sxlhv\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:48.606309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.606292    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-914d691e-sxlhv\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:48.697646 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.696496    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6866-d6274f" podUID="f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9" containerName="test-curl-6866-d6274f" containerID="cri-o://c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc" gracePeriod=2
Nov 04 04:00:48.707858 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.707835    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-914d691e-sxlhv\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:48.707941 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.707896    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-914d691e-sxlhv\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:48.707990 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.707945    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xws97\" (UniqueName: \"kubernetes.io/projected/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-kube-api-access-xws97\") pod \"maas-api-revoke-sub-914d691e-sxlhv\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:48.708544 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.708520    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-914d691e-sxlhv\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:48.710082 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.710064    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-914d691e-sxlhv\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:48.731705 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.731680    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xws97\" (UniqueName: \"kubernetes.io/projected/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-kube-api-access-xws97\") pod \"maas-api-revoke-sub-914d691e-sxlhv\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:48.836745 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:48.836694    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:49.118684 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:49.118655    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"]
Nov 04 04:00:49.120274 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:49.120248    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0ad55b_07ca_43c4_ac54_870f2baa82b0.slice/crio-308ddfeffae7f3466c8959574786135889f28d25a9613772cfa49336ad79815c WatchSource:0}: Error finding container 308ddfeffae7f3466c8959574786135889f28d25a9613772cfa49336ad79815c: Status 404 returned error can't find the container with id 308ddfeffae7f3466c8959574786135889f28d25a9613772cfa49336ad79815c
Nov 04 04:00:49.305459 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:49.305432    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:49.623057 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:49.623022    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:49.703152 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:49.703064    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv" event={"ID":"9c0ad55b-07ca-43c4-ac54-870f2baa82b0","Type":"ContainerStarted","Data":"bf6fa623bff26a2b065bd6b330680b3aaf7d738027dcf58f2557351e79f83ffa"}
Nov 04 04:00:49.703152 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:49.703116    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv" event={"ID":"9c0ad55b-07ca-43c4-ac54-870f2baa82b0","Type":"ContainerStarted","Data":"308ddfeffae7f3466c8959574786135889f28d25a9613772cfa49336ad79815c"}
Nov 04 04:00:50.486087 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:50.486054    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:50.711294 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:50.711261    6751 generic.go:356] "Generic (PLEG): container finished" podID="9c0ad55b-07ca-43c4-ac54-870f2baa82b0" containerID="bf6fa623bff26a2b065bd6b330680b3aaf7d738027dcf58f2557351e79f83ffa" exitCode=0
Nov 04 04:00:50.711869 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:50.711349    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv" event={"ID":"9c0ad55b-07ca-43c4-ac54-870f2baa82b0","Type":"ContainerDied","Data":"bf6fa623bff26a2b065bd6b330680b3aaf7d738027dcf58f2557351e79f83ffa"}
Nov 04 04:00:50.861727 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:50.861698    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6866-d6274f"
Nov 04 04:00:50.933826 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:50.933794    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9-kube-api-access-vgfnl\" (UniqueName: \"kubernetes.io/projected/f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9-kube-api-access-vgfnl\") pod \"f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9\" (UID: \"f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9\") "
Nov 04 04:00:50.935924 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:50.935889    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9-kube-api-access-vgfnl" pod "f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9" (UID: "f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9"). InnerVolumeSpecName "kube-api-access-vgfnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:00:51.035864 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.035773    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgfnl\" (UniqueName: \"kubernetes.io/projected/f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9-kube-api-access-vgfnl\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:51.088800 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:51.088764    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 04:00:51.300970 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.300875    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:51.501995 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.501958    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:51.608295 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.608263    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"]
Nov 04 04:00:51.612382 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.612340    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.626076 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.626006    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-64c20d\"/\"distinct-w4-64c20d-kserve-self-signed-certs\""
Nov 04 04:00:51.628393 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.628370    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"]
Nov 04 04:00:51.643443 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.643416    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/810c0166-1a84-4d90-9ec1-007dea6a282a-tls-certs\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.643584 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.643473    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-tmp-dir\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.643584 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.643539    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-model-cache\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.643704 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.643595    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-dshm\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.643858 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.643831    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s49d9\" (UniqueName: \"kubernetes.io/projected/810c0166-1a84-4d90-9ec1-007dea6a282a-kube-api-access-s49d9\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.643902 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.643886    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-home\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.718254 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.718215    6751 generic.go:356] "Generic (PLEG): container finished" podID="f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9" containerID="c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc" exitCode=137
Nov 04 04:00:51.718761 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.718262    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6866-d6274f"
Nov 04 04:00:51.718761 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.718313    6751 scope.go:122] "RemoveContainer" containerID="c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc"
Nov 04 04:00:51.748371 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.729133    6751 scope.go:122] "RemoveContainer" containerID="c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc"
Nov 04 04:00:51.748371 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:51.729451    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc\": container with ID starting with c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc not found: ID does not exist" containerID="c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc"
Nov 04 04:00:51.748371 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.729492    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc"} err="failed to get container status \"c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc\": rpc error: code = NotFound desc = could not find container \"c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc\": container with ID starting with c80c685150ec1ce3317e5f88eddcb0fee58e002ff6c155e2c1bd3e221e87c0dc not found: ID does not exist"
Nov 04 04:00:51.748371 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:00:51.729841    6751 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6866-d6274f\" is forbidden: User \"system:node:ip-10-0-129-253.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-253.us-east-2.compute.internal' and this object" podUID="f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9" pod="opendatahub/test-curl-6866-d6274f"
Nov 04 04:00:51.748371 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.745891    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/810c0166-1a84-4d90-9ec1-007dea6a282a-tls-certs\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.748371 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.745985    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-tmp-dir\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.748371 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.746017    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-model-cache\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.748371 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.746061    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-dshm\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.760309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.759797    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s49d9\" (UniqueName: \"kubernetes.io/projected/810c0166-1a84-4d90-9ec1-007dea6a282a-kube-api-access-s49d9\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.760309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.759863    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-home\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.760542 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.760518    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-home\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.771689 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.761520    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-model-cache\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.771689 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.763065    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:51.771689 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.763400    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-tmp-dir\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.806296 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.806258    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/810c0166-1a84-4d90-9ec1-007dea6a282a-tls-certs\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.806584 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.806519    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-dshm\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.811432 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.811361    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s49d9\" (UniqueName: \"kubernetes.io/projected/810c0166-1a84-4d90-9ec1-007dea6a282a-kube-api-access-s49d9\") pod \"distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") " pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.823188 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.814826    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9" path="/var/lib/kubelet/pods/f98d9cbe-fe5a-4982-9c8e-b8371b2b76f9/volumes"
Nov 04 04:00:51.823188 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.815215    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"]
Nov 04 04:00:51.830474 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.829825    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:51.830728 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.830710    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"]
Nov 04 04:00:51.927006 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.926922    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:51.961170 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.961140    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:51.963188 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.963159    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9wwf\" (UniqueName: \"kubernetes.io/projected/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-kube-api-access-x9wwf\") pod \"maas-api-revoke-sub-5cccef37-hxdpd\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:51.963309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.963210    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5cccef37-hxdpd\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:51.963348 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:51.963324    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5cccef37-hxdpd\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:52.064287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.064261    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-openshift-ca\") pod \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") "
Nov 04 04:00:52.064419 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.064340    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-xks-ca\") pod \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") "
Nov 04 04:00:52.064419 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.064384    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-kube-api-access-xws97\" (UniqueName: \"kubernetes.io/projected/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-kube-api-access-xws97\") pod \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\" (UID: \"9c0ad55b-07ca-43c4-ac54-870f2baa82b0\") "
Nov 04 04:00:52.064663 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.064638    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-openshift-ca" pod "9c0ad55b-07ca-43c4-ac54-870f2baa82b0" (UID: "9c0ad55b-07ca-43c4-ac54-870f2baa82b0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:00:52.064749 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.064732    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9wwf\" (UniqueName: \"kubernetes.io/projected/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-kube-api-access-x9wwf\") pod \"maas-api-revoke-sub-5cccef37-hxdpd\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:52.065075 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.064775    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5cccef37-hxdpd\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:52.065075 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.064851    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5cccef37-hxdpd\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:52.065075 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.064900    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:52.065491 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.065468    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5cccef37-hxdpd\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:52.066541 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.066514    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-xks-ca" pod "9c0ad55b-07ca-43c4-ac54-870f2baa82b0" (UID: "9c0ad55b-07ca-43c4-ac54-870f2baa82b0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:00:52.066672 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.066649    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-kube-api-access-xws97" pod "9c0ad55b-07ca-43c4-ac54-870f2baa82b0" (UID: "9c0ad55b-07ca-43c4-ac54-870f2baa82b0"). InnerVolumeSpecName "kube-api-access-xws97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:00:52.067134 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.067117    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5cccef37-hxdpd\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:52.076887 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.076869    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9wwf\" (UniqueName: \"kubernetes.io/projected/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-kube-api-access-x9wwf\") pod \"maas-api-revoke-sub-5cccef37-hxdpd\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:52.163313 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.163278    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:52.166063 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.166036    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:52.166063 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.166065    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xws97\" (UniqueName: \"kubernetes.io/projected/9c0ad55b-07ca-43c4-ac54-870f2baa82b0-kube-api-access-xws97\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:52.218699 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.218568    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"]
Nov 04 04:00:52.222424 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:52.222391    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod810c0166_1a84_4d90_9ec1_007dea6a282a.slice/crio-82821c8b2f30c9a77b5c7e287092debc9f306e85dddc7cd5cacf7cf3ad39ad19 WatchSource:0}: Error finding container 82821c8b2f30c9a77b5c7e287092debc9f306e85dddc7cd5cacf7cf3ad39ad19: Status 404 returned error can't find the container with id 82821c8b2f30c9a77b5c7e287092debc9f306e85dddc7cd5cacf7cf3ad39ad19
Nov 04 04:00:52.306990 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.305092    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:52.453449 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.453425    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"]
Nov 04 04:00:52.455547 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:52.455520    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacc5759a_fd5e_4e1a_94ef_57a90e9806a9.slice/crio-292669d9ede01845a4678e07b63eadad0b5da6701bf7ff587b0f0e77e517c903 WatchSource:0}: Error finding container 292669d9ede01845a4678e07b63eadad0b5da6701bf7ff587b0f0e77e517c903: Status 404 returned error can't find the container with id 292669d9ede01845a4678e07b63eadad0b5da6701bf7ff587b0f0e77e517c903
Nov 04 04:00:52.725810 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.725739    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd" event={"ID":"acc5759a-fd5e-4e1a-94ef-57a90e9806a9","Type":"ContainerStarted","Data":"292669d9ede01845a4678e07b63eadad0b5da6701bf7ff587b0f0e77e517c903"}
Nov 04 04:00:52.726990 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.726966    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv"
Nov 04 04:00:52.727087 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.726963    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-914d691e-sxlhv" event={"ID":"9c0ad55b-07ca-43c4-ac54-870f2baa82b0","Type":"ContainerDied","Data":"308ddfeffae7f3466c8959574786135889f28d25a9613772cfa49336ad79815c"}
Nov 04 04:00:52.727087 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.727083    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="308ddfeffae7f3466c8959574786135889f28d25a9613772cfa49336ad79815c"
Nov 04 04:00:52.728553 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.728525    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz" event={"ID":"810c0166-1a84-4d90-9ec1-007dea6a282a","Type":"ContainerStarted","Data":"2b51f6e6eb9c5fd7fc45dc421be2d3bcd7310ef9f94c54d19b8678605081d65d"}
Nov 04 04:00:52.728668 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.728563    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz" event={"ID":"810c0166-1a84-4d90-9ec1-007dea6a282a","Type":"ContainerStarted","Data":"82821c8b2f30c9a77b5c7e287092debc9f306e85dddc7cd5cacf7cf3ad39ad19"}
Nov 04 04:00:52.752487 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.752440    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz" podStartSLOduration=1.752428975 podStartE2EDuration="1.752428975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:52.749980195 +0000 UTC m=+647.083918078" watchObservedRunningTime="2026-11-04 04:00:52.752428975 +0000 UTC m=+647.086366852"
Nov 04 04:00:52.927501 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:52.927452    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:00:53.114200 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:53.114165    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"]
Nov 04 04:00:53.284277 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:53.284245    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:53.507823 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:53.507755    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:53.735468 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:53.735434    6751 generic.go:356] "Generic (PLEG): container finished" podID="acc5759a-fd5e-4e1a-94ef-57a90e9806a9" containerID="1a31f2be6962490a81a90f9095629bb3911b08bd617310fd44c7170c99c95a33" exitCode=0
Nov 04 04:00:53.735852 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:53.735522    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd" event={"ID":"acc5759a-fd5e-4e1a-94ef-57a90e9806a9","Type":"ContainerDied","Data":"1a31f2be6962490a81a90f9095629bb3911b08bd617310fd44c7170c99c95a33"}
Nov 04 04:00:53.913931 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:53.913900    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:54.142705 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.142671    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:54.283699 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.275514    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"]
Nov 04 04:00:54.296170 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.291802    6751 cpu_manager.go:414] "containerMap: removing container" podUID="9c0ad55b-07ca-43c4-ac54-870f2baa82b0" containerName="revoke-keys"
Nov 04 04:00:54.296170 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.291846    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c0ad55b-07ca-43c4-ac54-870f2baa82b0" containerName="revoke-keys"
Nov 04 04:00:54.296170 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.291992    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c0ad55b-07ca-43c4-ac54-870f2baa82b0" containerName="revoke-keys"
Nov 04 04:00:54.302645 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.302390    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"]
Nov 04 04:00:54.302645 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.302533    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.337641 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.337600    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cd9fe1a4-88lrc\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.337785 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.337683    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtjhf\" (UniqueName: \"kubernetes.io/projected/62c7a280-ae91-4bca-8370-5e1d88cb9745-kube-api-access-mtjhf\") pod \"maas-api-revoke-sub-cd9fe1a4-88lrc\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.337859 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.337830    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cd9fe1a4-88lrc\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.373296 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.373271    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:54.438898 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.438875    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtjhf\" (UniqueName: \"kubernetes.io/projected/62c7a280-ae91-4bca-8370-5e1d88cb9745-kube-api-access-mtjhf\") pod \"maas-api-revoke-sub-cd9fe1a4-88lrc\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.438996 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.438931    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cd9fe1a4-88lrc\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.439059 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.439002    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cd9fe1a4-88lrc\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.439758 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.439726    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cd9fe1a4-88lrc\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.441303 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.441283    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cd9fe1a4-88lrc\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.451035 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.451008    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtjhf\" (UniqueName: \"kubernetes.io/projected/62c7a280-ae91-4bca-8370-5e1d88cb9745-kube-api-access-mtjhf\") pod \"maas-api-revoke-sub-cd9fe1a4-88lrc\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.622268 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.622233    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:54.881423 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.881357    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:54.919343 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.919315    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"]
Nov 04 04:00:54.921737 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:54.921702    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c7a280_ae91_4bca_8370_5e1d88cb9745.slice/crio-065e895b3b6cd4eeb5d12f42e93a35efbfacf32cd9121712efed155211436cbb WatchSource:0}: Error finding container 065e895b3b6cd4eeb5d12f42e93a35efbfacf32cd9121712efed155211436cbb: Status 404 returned error can't find the container with id 065e895b3b6cd4eeb5d12f42e93a35efbfacf32cd9121712efed155211436cbb
Nov 04 04:00:54.945351 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.945322    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-xks-ca\") pod \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") "
Nov 04 04:00:54.945485 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.945382    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-openshift-ca\") pod \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") "
Nov 04 04:00:54.945485 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.945464    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-kube-api-access-x9wwf\" (UniqueName: \"kubernetes.io/projected/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-kube-api-access-x9wwf\") pod \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\" (UID: \"acc5759a-fd5e-4e1a-94ef-57a90e9806a9\") "
Nov 04 04:00:54.945850 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.945821    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-openshift-ca" pod "acc5759a-fd5e-4e1a-94ef-57a90e9806a9" (UID: "acc5759a-fd5e-4e1a-94ef-57a90e9806a9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:00:54.946124 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.946098    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:54.947524 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.947502    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-xks-ca" pod "acc5759a-fd5e-4e1a-94ef-57a90e9806a9" (UID: "acc5759a-fd5e-4e1a-94ef-57a90e9806a9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:00:54.947945 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:54.947925    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-kube-api-access-x9wwf" pod "acc5759a-fd5e-4e1a-94ef-57a90e9806a9" (UID: "acc5759a-fd5e-4e1a-94ef-57a90e9806a9"). InnerVolumeSpecName "kube-api-access-x9wwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:00:55.047156 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:55.047120    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:55.047156 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:55.047149    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9wwf\" (UniqueName: \"kubernetes.io/projected/acc5759a-fd5e-4e1a-94ef-57a90e9806a9-kube-api-access-x9wwf\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:55.435227 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:55.435146    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:55.748987 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:55.748895    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc" event={"ID":"62c7a280-ae91-4bca-8370-5e1d88cb9745","Type":"ContainerStarted","Data":"99f4695b25402e98741024f3acdcd36a7f8f3f42e81b255bda5077e6d79797bb"}
Nov 04 04:00:55.749162 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:55.748989    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc" event={"ID":"62c7a280-ae91-4bca-8370-5e1d88cb9745","Type":"ContainerStarted","Data":"065e895b3b6cd4eeb5d12f42e93a35efbfacf32cd9121712efed155211436cbb"}
Nov 04 04:00:55.750147 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:55.750113    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd" event={"ID":"acc5759a-fd5e-4e1a-94ef-57a90e9806a9","Type":"ContainerDied","Data":"292669d9ede01845a4678e07b63eadad0b5da6701bf7ff587b0f0e77e517c903"}
Nov 04 04:00:55.750147 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:55.750138    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5cccef37-hxdpd"
Nov 04 04:00:55.750330 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:55.750161    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="292669d9ede01845a4678e07b63eadad0b5da6701bf7ff587b0f0e77e517c903"
Nov 04 04:00:55.769946 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:55.769902    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc" podStartSLOduration=1.769890301 podStartE2EDuration="1.769890301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:00:55.767445861 +0000 UTC m=+650.101383739" watchObservedRunningTime="2026-11-04 04:00:55.769890301 +0000 UTC m=+650.103828179"
Nov 04 04:00:56.378781 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:56.378745    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"]
Nov 04 04:00:56.394682 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:56.390954    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"]
Nov 04 04:00:56.413580 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:56.411016    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:56.614362 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:56.614332    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:56.756985 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:56.756904    6751 generic.go:356] "Generic (PLEG): container finished" podID="62c7a280-ae91-4bca-8370-5e1d88cb9745" containerID="99f4695b25402e98741024f3acdcd36a7f8f3f42e81b255bda5077e6d79797bb" exitCode=0
Nov 04 04:00:56.757110 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:56.756988    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc" event={"ID":"62c7a280-ae91-4bca-8370-5e1d88cb9745","Type":"ContainerDied","Data":"99f4695b25402e98741024f3acdcd36a7f8f3f42e81b255bda5077e6d79797bb"}
Nov 04 04:00:56.816666 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:56.816639    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:57.089420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.089336    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"]
Nov 04 04:00:57.089854 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.089837    6751 cpu_manager.go:414] "containerMap: removing container" podUID="acc5759a-fd5e-4e1a-94ef-57a90e9806a9" containerName="revoke-keys"
Nov 04 04:00:57.089854 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.089852    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acc5759a-fd5e-4e1a-94ef-57a90e9806a9" containerName="revoke-keys"
Nov 04 04:00:57.090001 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.089940    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="acc5759a-fd5e-4e1a-94ef-57a90e9806a9" containerName="revoke-keys"
Nov 04 04:00:57.093160 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.093144    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.107052 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.107018    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"]
Nov 04 04:00:57.171183 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.171152    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0539c1b2-tlmfj\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.171345 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.171194    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf9dr\" (UniqueName: \"kubernetes.io/projected/9073d5de-a959-496e-94e0-4ca6ecf2d37c-kube-api-access-nf9dr\") pod \"maas-api-revoke-sub-0539c1b2-tlmfj\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.171386 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.171334    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0539c1b2-tlmfj\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.278610 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.273549    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0539c1b2-tlmfj\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.278610 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.273613    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf9dr\" (UniqueName: \"kubernetes.io/projected/9073d5de-a959-496e-94e0-4ca6ecf2d37c-kube-api-access-nf9dr\") pod \"maas-api-revoke-sub-0539c1b2-tlmfj\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.278610 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.273710    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0539c1b2-tlmfj\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.278610 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.274635    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0539c1b2-tlmfj\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.295667 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.286251    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0539c1b2-tlmfj\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.295667 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.294400    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf9dr\" (UniqueName: \"kubernetes.io/projected/9073d5de-a959-496e-94e0-4ca6ecf2d37c-kube-api-access-nf9dr\") pod \"maas-api-revoke-sub-0539c1b2-tlmfj\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.320358 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.320326    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:57.407261 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.407235    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:00:57.684261 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.684226    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"]
Nov 04 04:00:57.686493 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:00:57.686467    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9073d5de_a959_496e_94e0_4ca6ecf2d37c.slice/crio-4f1b5f9df8c433a1081da4742571c7b66822edd5e5c9435ff773ecb3593049bd WatchSource:0}: Error finding container 4f1b5f9df8c433a1081da4742571c7b66822edd5e5c9435ff773ecb3593049bd: Status 404 returned error can't find the container with id 4f1b5f9df8c433a1081da4742571c7b66822edd5e5c9435ff773ecb3593049bd
Nov 04 04:00:57.764844 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.764816    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj" event={"ID":"9073d5de-a959-496e-94e0-4ca6ecf2d37c","Type":"ContainerStarted","Data":"4f1b5f9df8c433a1081da4742571c7b66822edd5e5c9435ff773ecb3593049bd"}
Nov 04 04:00:57.903487 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.903466    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:57.983792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.983765    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62c7a280-ae91-4bca-8370-5e1d88cb9745-kube-api-access-mtjhf\" (UniqueName: \"kubernetes.io/projected/62c7a280-ae91-4bca-8370-5e1d88cb9745-kube-api-access-mtjhf\") pod \"62c7a280-ae91-4bca-8370-5e1d88cb9745\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") "
Nov 04 04:00:57.983899 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.983811    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-xks-ca\") pod \"62c7a280-ae91-4bca-8370-5e1d88cb9745\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") "
Nov 04 04:00:57.983899 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.983885    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-openshift-ca\") pod \"62c7a280-ae91-4bca-8370-5e1d88cb9745\" (UID: \"62c7a280-ae91-4bca-8370-5e1d88cb9745\") "
Nov 04 04:00:57.984254 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.984226    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-openshift-ca" pod "62c7a280-ae91-4bca-8370-5e1d88cb9745" (UID: "62c7a280-ae91-4bca-8370-5e1d88cb9745"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:00:57.985769 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.985744    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-xks-ca" pod "62c7a280-ae91-4bca-8370-5e1d88cb9745" (UID: "62c7a280-ae91-4bca-8370-5e1d88cb9745"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:00:57.985888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:57.985787    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62c7a280-ae91-4bca-8370-5e1d88cb9745-kube-api-access-mtjhf" pod "62c7a280-ae91-4bca-8370-5e1d88cb9745" (UID: "62c7a280-ae91-4bca-8370-5e1d88cb9745"). InnerVolumeSpecName "kube-api-access-mtjhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:00:58.085420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.085392    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtjhf\" (UniqueName: \"kubernetes.io/projected/62c7a280-ae91-4bca-8370-5e1d88cb9745-kube-api-access-mtjhf\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:58.085420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.085417    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:58.085597 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.085428    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/62c7a280-ae91-4bca-8370-5e1d88cb9745-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:00:58.265305 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.265228    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:58.556379 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.556285    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:58.765820 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.765784    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"]
Nov 04 04:00:58.772392 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.772361    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"]
Nov 04 04:00:58.774378 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.774349    6751 generic.go:356] "Generic (PLEG): container finished" podID="9073d5de-a959-496e-94e0-4ca6ecf2d37c" containerID="1d29517013c904ac5addbdb315a354c721f4a3ada664f6d71dd9e891deed793c" exitCode=0
Nov 04 04:00:58.774491 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.774440    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj" event={"ID":"9073d5de-a959-496e-94e0-4ca6ecf2d37c","Type":"ContainerDied","Data":"1d29517013c904ac5addbdb315a354c721f4a3ada664f6d71dd9e891deed793c"}
Nov 04 04:00:58.775891 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.775869    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc" event={"ID":"62c7a280-ae91-4bca-8370-5e1d88cb9745","Type":"ContainerDied","Data":"065e895b3b6cd4eeb5d12f42e93a35efbfacf32cd9121712efed155211436cbb"}
Nov 04 04:00:58.775980 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.775897    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd9fe1a4-88lrc"
Nov 04 04:00:58.775980 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.775906    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="065e895b3b6cd4eeb5d12f42e93a35efbfacf32cd9121712efed155211436cbb"
Nov 04 04:00:58.971016 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:58.970467    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:59.180402 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:59.180374    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:59.702479 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:59.702450    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:00:59.927435 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:00:59.927410    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:01:00.006720 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.006646    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9073d5de-a959-496e-94e0-4ca6ecf2d37c-kube-api-access-nf9dr\" (UniqueName: \"kubernetes.io/projected/9073d5de-a959-496e-94e0-4ca6ecf2d37c-kube-api-access-nf9dr\") pod \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") "
Nov 04 04:01:00.006875 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.006732    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-openshift-ca\") pod \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") "
Nov 04 04:01:00.006875 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.006785    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-xks-ca\") pod \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\" (UID: \"9073d5de-a959-496e-94e0-4ca6ecf2d37c\") "
Nov 04 04:01:00.007130 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.007109    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-openshift-ca" pod "9073d5de-a959-496e-94e0-4ca6ecf2d37c" (UID: "9073d5de-a959-496e-94e0-4ca6ecf2d37c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:00.007329 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.007313    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:00.008824 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.008798    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9073d5de-a959-496e-94e0-4ca6ecf2d37c-kube-api-access-nf9dr" pod "9073d5de-a959-496e-94e0-4ca6ecf2d37c" (UID: "9073d5de-a959-496e-94e0-4ca6ecf2d37c"). InnerVolumeSpecName "kube-api-access-nf9dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:00.009060 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.009046    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-xks-ca" pod "9073d5de-a959-496e-94e0-4ca6ecf2d37c" (UID: "9073d5de-a959-496e-94e0-4ca6ecf2d37c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:00.014534 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.014511    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"]
Nov 04 04:01:00.015007 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.014994    6751 cpu_manager.go:414] "containerMap: removing container" podUID="9073d5de-a959-496e-94e0-4ca6ecf2d37c" containerName="revoke-keys"
Nov 04 04:01:00.015050 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.015009    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9073d5de-a959-496e-94e0-4ca6ecf2d37c" containerName="revoke-keys"
Nov 04 04:01:00.015050 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.015035    6751 cpu_manager.go:414] "containerMap: removing container" podUID="62c7a280-ae91-4bca-8370-5e1d88cb9745" containerName="revoke-keys"
Nov 04 04:01:00.015050 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.015040    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62c7a280-ae91-4bca-8370-5e1d88cb9745" containerName="revoke-keys"
Nov 04 04:01:00.015147 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.015112    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="62c7a280-ae91-4bca-8370-5e1d88cb9745" containerName="revoke-keys"
Nov 04 04:01:00.015147 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.015125    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="9073d5de-a959-496e-94e0-4ca6ecf2d37c" containerName="revoke-keys"
Nov 04 04:01:00.018421 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.018405    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.035791 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.035767    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-56a44f\"/\"kube-root-ca.crt\""
Nov 04 04:01:00.035903 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.035863    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"]
Nov 04 04:01:00.038331 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.038310    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-56a44f\"/\"openshift-service-ca.crt\""
Nov 04 04:01:00.038398 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.038354    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-56a44f\"/\"default-dockercfg-vfh6g\""
Nov 04 04:01:00.038451 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.038397    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-56a44f\"/\"distinct-2-w5-56a44f-kserve-self-signed-certs\""
Nov 04 04:01:00.108590 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.108563    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tmp-dir\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.108708 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.108601    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tls-certs\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.108708 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.108692    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-model-cache\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.108804 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.108783    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-dshm\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.108879 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.108860    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-home\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.108929 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.108915    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6t7b\" (UniqueName: \"kubernetes.io/projected/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-kube-api-access-n6t7b\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.109040 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.109025    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf9dr\" (UniqueName: \"kubernetes.io/projected/9073d5de-a959-496e-94e0-4ca6ecf2d37c-kube-api-access-nf9dr\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:00.109074 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.109046    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9073d5de-a959-496e-94e0-4ca6ecf2d37c-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:00.172683 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.169638    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:00.209739 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.209706    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-dshm\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.209915 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.209862    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-home\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.209915 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.209894    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6t7b\" (UniqueName: \"kubernetes.io/projected/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-kube-api-access-n6t7b\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.210098 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.209980    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tmp-dir\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.210098 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.210024    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tls-certs\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.210098 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.210081    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-model-cache\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.210342 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.210320    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-home\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.210408 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.210339    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tmp-dir\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.210468 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.210406    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-model-cache\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.212546 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.212521    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-dshm\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.212685 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.212666    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tls-certs\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.228357 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.228333    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6t7b\" (UniqueName: \"kubernetes.io/projected/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-kube-api-access-n6t7b\") pod \"distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") " pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.339420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.339354    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:00.624055 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.624021    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"]
Nov 04 04:01:00.661016 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:00.654411    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70f1bcac_af6a_4019_8fbe_f8a9eb587ba0.slice/crio-78e2313540ba21b43fbcb8eb9095ee6d6be6a5667d9f7a16884a265306306198 WatchSource:0}: Error finding container 78e2313540ba21b43fbcb8eb9095ee6d6be6a5667d9f7a16884a265306306198: Status 404 returned error can't find the container with id 78e2313540ba21b43fbcb8eb9095ee6d6be6a5667d9f7a16884a265306306198
Nov 04 04:01:00.678457 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.678429    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:00.788581 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.788555    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj"
Nov 04 04:01:00.789009 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.788590    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0539c1b2-tlmfj" event={"ID":"9073d5de-a959-496e-94e0-4ca6ecf2d37c","Type":"ContainerDied","Data":"4f1b5f9df8c433a1081da4742571c7b66822edd5e5c9435ff773ecb3593049bd"}
Nov 04 04:01:00.789009 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.788646    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f1b5f9df8c433a1081da4742571c7b66822edd5e5c9435ff773ecb3593049bd"
Nov 04 04:01:00.790351 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.790329    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj" event={"ID":"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0","Type":"ContainerStarted","Data":"8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef"}
Nov 04 04:01:00.790460 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.790362    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj" event={"ID":"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0","Type":"ContainerStarted","Data":"78e2313540ba21b43fbcb8eb9095ee6d6be6a5667d9f7a16884a265306306198"}
Nov 04 04:01:00.815030 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:00.814987    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj" podStartSLOduration=1.814971914 podStartE2EDuration="1.814971914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:00.814582439 +0000 UTC m=+655.148520318" watchObservedRunningTime="2026-11-04 04:01:00.814971914 +0000 UTC m=+655.148909791"
Nov 04 04:01:01.339786 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:01.339752    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:01.589293 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:01.589257    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:01.846268 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:01.846237    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:01.927774 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:01.927749    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:01:01.940336 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:01.940313    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:01:02.050487 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:02.050433    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:02.483473 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:02.483442    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:02.819846 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:02.819765    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:01:03.190931 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.190892    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"]
Nov 04 04:01:03.195361 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.195336    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.202881 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.202847    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 04:01:03.202881 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.202866    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 04:01:03.203071 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.202898    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2wnx7\""
Nov 04 04:01:03.203071 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.202898    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 04:01:03.203181 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.203162    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"]
Nov 04 04:01:03.260649 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.257470    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc076113-rqmv8\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.260649 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.257601    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc076113-rqmv8\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.260649 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.257665    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sbsn\" (UniqueName: \"kubernetes.io/projected/f093ec8c-4edd-48d9-aace-3139fa885014-kube-api-access-2sbsn\") pod \"maas-api-revoke-sub-cc076113-rqmv8\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.306609 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.306564    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:03.358607 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.358575    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc076113-rqmv8\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.358771 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.358655    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sbsn\" (UniqueName: \"kubernetes.io/projected/f093ec8c-4edd-48d9-aace-3139fa885014-kube-api-access-2sbsn\") pod \"maas-api-revoke-sub-cc076113-rqmv8\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.358829 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.358813    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc076113-rqmv8\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.359275 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.359255    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc076113-rqmv8\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.361232 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.361210    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc076113-rqmv8\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.370874 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.370855    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sbsn\" (UniqueName: \"kubernetes.io/projected/f093ec8c-4edd-48d9-aace-3139fa885014-kube-api-access-2sbsn\") pod \"maas-api-revoke-sub-cc076113-rqmv8\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.514024 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.513940    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:03.795285 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.795257    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"]
Nov 04 04:01:03.796304 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:03.796280    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf093ec8c_4edd_48d9_aace_3139fa885014.slice/crio-66127931f490bbb9e513772153891710ee8cd96ad068d5b6266127e89515dabe WatchSource:0}: Error finding container 66127931f490bbb9e513772153891710ee8cd96ad068d5b6266127e89515dabe: Status 404 returned error can't find the container with id 66127931f490bbb9e513772153891710ee8cd96ad068d5b6266127e89515dabe
Nov 04 04:01:03.817021 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.816993    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8" event={"ID":"f093ec8c-4edd-48d9-aace-3139fa885014","Type":"ContainerStarted","Data":"66127931f490bbb9e513772153891710ee8cd96ad068d5b6266127e89515dabe"}
Nov 04 04:01:03.942638 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:03.937991    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:04.166265 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:04.166219    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:04.376666 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:04.376608    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:04.787880 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:04.787849    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:04.823554 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:04.823532    6751 generic.go:356] "Generic (PLEG): container finished" podID="f093ec8c-4edd-48d9-aace-3139fa885014" containerID="d6e6900cb64f6d2f44f65b92733e5e9b2e8d0675cbcb34d785e782e0f45f0312" exitCode=0
Nov 04 04:01:04.823672 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:04.823650    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8" event={"ID":"f093ec8c-4edd-48d9-aace-3139fa885014","Type":"ContainerDied","Data":"d6e6900cb64f6d2f44f65b92733e5e9b2e8d0675cbcb34d785e782e0f45f0312"}
Nov 04 04:01:04.996189 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:04.996115    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:05.209046 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:05.209010    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:05.910782 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:05.910508    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:05.983323 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:05.983301    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:06.126825 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.111884    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f093ec8c-4edd-48d9-aace-3139fa885014-kube-api-access-2sbsn\" (UniqueName: \"kubernetes.io/projected/f093ec8c-4edd-48d9-aace-3139fa885014-kube-api-access-2sbsn\") pod \"f093ec8c-4edd-48d9-aace-3139fa885014\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") "
Nov 04 04:01:06.126825 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.112013    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-xks-ca\") pod \"f093ec8c-4edd-48d9-aace-3139fa885014\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") "
Nov 04 04:01:06.126825 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.112128    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-openshift-ca\") pod \"f093ec8c-4edd-48d9-aace-3139fa885014\" (UID: \"f093ec8c-4edd-48d9-aace-3139fa885014\") "
Nov 04 04:01:06.167652 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.158381    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-xks-ca" pod "f093ec8c-4edd-48d9-aace-3139fa885014" (UID: "f093ec8c-4edd-48d9-aace-3139fa885014"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:06.168152 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.168112    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f093ec8c-4edd-48d9-aace-3139fa885014-kube-api-access-2sbsn" pod "f093ec8c-4edd-48d9-aace-3139fa885014" (UID: "f093ec8c-4edd-48d9-aace-3139fa885014"). InnerVolumeSpecName "kube-api-access-2sbsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:06.174331 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.173451    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-openshift-ca" pod "f093ec8c-4edd-48d9-aace-3139fa885014" (UID: "f093ec8c-4edd-48d9-aace-3139fa885014"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:06.175150 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.175113    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:06.238644 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.234700    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sbsn\" (UniqueName: \"kubernetes.io/projected/f093ec8c-4edd-48d9-aace-3139fa885014-kube-api-access-2sbsn\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:06.238886 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.238864    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:06.239017 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.239001    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f093ec8c-4edd-48d9-aace-3139fa885014-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:06.847531 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.847387    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8" event={"ID":"f093ec8c-4edd-48d9-aace-3139fa885014","Type":"ContainerDied","Data":"66127931f490bbb9e513772153891710ee8cd96ad068d5b6266127e89515dabe"}
Nov 04 04:01:06.847531 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.847456    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66127931f490bbb9e513772153891710ee8cd96ad068d5b6266127e89515dabe"
Nov 04 04:01:06.847531 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.847555    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc076113-rqmv8"
Nov 04 04:01:06.902742 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:06.902710    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:07.134691 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.134052    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"]
Nov 04 04:01:07.227642 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.227449    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:07.465508 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.465406    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:07.666414 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.666378    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"]
Nov 04 04:01:07.667117 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.667097    6751 cpu_manager.go:414] "containerMap: removing container" podUID="f093ec8c-4edd-48d9-aace-3139fa885014" containerName="revoke-keys"
Nov 04 04:01:07.667162 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.667121    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f093ec8c-4edd-48d9-aace-3139fa885014" containerName="revoke-keys"
Nov 04 04:01:07.667337 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.667322    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="f093ec8c-4edd-48d9-aace-3139fa885014" containerName="revoke-keys"
Nov 04 04:01:07.672915 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.672894    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:07.679655 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.678643    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"]
Nov 04 04:01:07.680263 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.680243    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 04:01:07.680754 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.680464    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 04:01:07.680754 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.680599    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2wnx7\""
Nov 04 04:01:07.680915 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.680701    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 04:01:07.796988 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.796911    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crqcg\" (UniqueName: \"kubernetes.io/projected/19163451-75a0-4fe0-a8c4-97c280b2ebd7-kube-api-access-crqcg\") pod \"maas-api-revoke-sub-d1017559-g47dg\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:07.796988 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.796979    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d1017559-g47dg\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:07.797204 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.797153    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d1017559-g47dg\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:07.898562 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.898524    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crqcg\" (UniqueName: \"kubernetes.io/projected/19163451-75a0-4fe0-a8c4-97c280b2ebd7-kube-api-access-crqcg\") pod \"maas-api-revoke-sub-d1017559-g47dg\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:07.898562 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.898566    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d1017559-g47dg\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:07.898826 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.898742    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d1017559-g47dg\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:07.899510 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.899480    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d1017559-g47dg\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:07.901417 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.901389    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d1017559-g47dg\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:07.910840 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.910812    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crqcg\" (UniqueName: \"kubernetes.io/projected/19163451-75a0-4fe0-a8c4-97c280b2ebd7-kube-api-access-crqcg\") pod \"maas-api-revoke-sub-d1017559-g47dg\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:07.994685 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:07.994652    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:08.099939 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.099824    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:08.135409 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.134887    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj" podUID="03acbcbe-3995-4e05-bf6e-8430b273e0b7" containerName="main" containerID="cri-o://55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34" gracePeriod=45
Nov 04 04:01:08.336202 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.335416    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"]
Nov 04 04:01:08.340456 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:08.338076    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19163451_75a0_4fe0_a8c4_97c280b2ebd7.slice/crio-05d60a0ec3633cb6926b4a0fe83fdf018160b97754095a5a7f5d067c10384942 WatchSource:0}: Error finding container 05d60a0ec3633cb6926b4a0fe83fdf018160b97754095a5a7f5d067c10384942: Status 404 returned error can't find the container with id 05d60a0ec3633cb6926b4a0fe83fdf018160b97754095a5a7f5d067c10384942
Nov 04 04:01:08.400257 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.400238    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 04:01:08.504906 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.504876    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tmp-dir\") pod \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") "
Nov 04 04:01:08.505059 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.504933    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-dshm\") pod \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") "
Nov 04 04:01:08.505059 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.505022    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-home\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-home\") pod \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") "
Nov 04 04:01:08.505177 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.505082    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tls-certs\" (UniqueName: \"kubernetes.io/secret/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tls-certs\") pod \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") "
Nov 04 04:01:08.505177 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.505123    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-model-cache\") pod \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") "
Nov 04 04:01:08.505177 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.505131    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tmp-dir" pod "03acbcbe-3995-4e05-bf6e-8430b273e0b7" (UID: "03acbcbe-3995-4e05-bf6e-8430b273e0b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:08.505177 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.505153    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03acbcbe-3995-4e05-bf6e-8430b273e0b7-kube-api-access-b98qv\" (UniqueName: \"kubernetes.io/projected/03acbcbe-3995-4e05-bf6e-8430b273e0b7-kube-api-access-b98qv\") pod \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\" (UID: \"03acbcbe-3995-4e05-bf6e-8430b273e0b7\") "
Nov 04 04:01:08.505378 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.505320    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-home" pod "03acbcbe-3995-4e05-bf6e-8430b273e0b7" (UID: "03acbcbe-3995-4e05-bf6e-8430b273e0b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:08.505378 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.505358    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-model-cache" pod "03acbcbe-3995-4e05-bf6e-8430b273e0b7" (UID: "03acbcbe-3995-4e05-bf6e-8430b273e0b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:08.505577 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.505560    6751 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tmp-dir\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:08.505661 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.505578    6751 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-home\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:08.505661 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.505588    6751 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-model-cache\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:08.550484 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.550448    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tls-certs" pod "03acbcbe-3995-4e05-bf6e-8430b273e0b7" (UID: "03acbcbe-3995-4e05-bf6e-8430b273e0b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:08.550612 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.550495    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03acbcbe-3995-4e05-bf6e-8430b273e0b7-kube-api-access-b98qv" pod "03acbcbe-3995-4e05-bf6e-8430b273e0b7" (UID: "03acbcbe-3995-4e05-bf6e-8430b273e0b7"). InnerVolumeSpecName "kube-api-access-b98qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:08.550612 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.550501    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-dshm" pod "03acbcbe-3995-4e05-bf6e-8430b273e0b7" (UID: "03acbcbe-3995-4e05-bf6e-8430b273e0b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:08.607974 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.607102    6751 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03acbcbe-3995-4e05-bf6e-8430b273e0b7-tls-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:08.607974 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.607132    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b98qv\" (UniqueName: \"kubernetes.io/projected/03acbcbe-3995-4e05-bf6e-8430b273e0b7-kube-api-access-b98qv\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:08.607974 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.607156    6751 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03acbcbe-3995-4e05-bf6e-8430b273e0b7-dshm\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:08.868503 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.868395    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg" event={"ID":"19163451-75a0-4fe0-a8c4-97c280b2ebd7","Type":"ContainerStarted","Data":"bdd3a939810b111008342e2e261b062fbd74ac06667ea29eb1503f0c0255311b"}
Nov 04 04:01:08.868503 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.868460    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg" event={"ID":"19163451-75a0-4fe0-a8c4-97c280b2ebd7","Type":"ContainerStarted","Data":"05d60a0ec3633cb6926b4a0fe83fdf018160b97754095a5a7f5d067c10384942"}
Nov 04 04:01:08.869802 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.869775    6751 generic.go:356] "Generic (PLEG): container finished" podID="03acbcbe-3995-4e05-bf6e-8430b273e0b7" containerID="55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34" exitCode=0
Nov 04 04:01:08.869857 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.869828    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"
Nov 04 04:01:08.869897 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.869857    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj" event={"ID":"03acbcbe-3995-4e05-bf6e-8430b273e0b7","Type":"ContainerDied","Data":"55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34"}
Nov 04 04:01:08.869931 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.869899    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj" event={"ID":"03acbcbe-3995-4e05-bf6e-8430b273e0b7","Type":"ContainerDied","Data":"46d398cbd8e20d053a99dde8d6f85a4cdda0421131c7da7a74f37536d3688195"}
Nov 04 04:01:08.869931 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.869919    6751 scope.go:122] "RemoveContainer" containerID="55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34"
Nov 04 04:01:08.879963 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.879948    6751 scope.go:122] "RemoveContainer" containerID="55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34"
Nov 04 04:01:08.880216 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:08.880185    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34\": container with ID starting with 55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34 not found: ID does not exist" containerID="55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34"
Nov 04 04:01:08.880283 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.880222    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34"} err="failed to get container status \"55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34\": rpc error: code = NotFound desc = could not find container \"55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34\": container with ID starting with 55a0424a54d615fb4abeeea24abf72833e8f00a88f6df47c7f53ef962b0a7f34 not found: ID does not exist"
Nov 04 04:01:08.892190 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.892152    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg" podStartSLOduration=1.892141724 podStartE2EDuration="1.892141724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:08.890164868 +0000 UTC m=+663.224102747" watchObservedRunningTime="2026-11-04 04:01:08.892141724 +0000 UTC m=+663.226079602"
Nov 04 04:01:08.899839 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.899820    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:08.904977 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.904956    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"]
Nov 04 04:01:08.909245 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:08.909222    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-998997/simulator-w3-998997-kserve-6bbb58c68f-lc7nj"]
Nov 04 04:01:09.317012 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:09.316990    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:09.586706 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:09.584731    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:09.784977 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:09.784947    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03acbcbe-3995-4e05-bf6e-8430b273e0b7" path="/var/lib/kubelet/pods/03acbcbe-3995-4e05-bf6e-8430b273e0b7/volumes"
Nov 04 04:01:09.792547 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:09.792522    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:09.876283 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:09.876255    6751 generic.go:356] "Generic (PLEG): container finished" podID="19163451-75a0-4fe0-a8c4-97c280b2ebd7" containerID="bdd3a939810b111008342e2e261b062fbd74ac06667ea29eb1503f0c0255311b" exitCode=0
Nov 04 04:01:09.876420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:09.876336    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg" event={"ID":"19163451-75a0-4fe0-a8c4-97c280b2ebd7","Type":"ContainerDied","Data":"bdd3a939810b111008342e2e261b062fbd74ac06667ea29eb1503f0c0255311b"}
Nov 04 04:01:10.339961 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.339880    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:10.352357 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.352333    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:10.386546 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.386517    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"]
Nov 04 04:01:10.387014 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.386992    6751 cpu_manager.go:414] "containerMap: removing container" podUID="03acbcbe-3995-4e05-bf6e-8430b273e0b7" containerName="main"
Nov 04 04:01:10.387014 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.387008    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03acbcbe-3995-4e05-bf6e-8430b273e0b7" containerName="main"
Nov 04 04:01:10.387129 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.387110    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="03acbcbe-3995-4e05-bf6e-8430b273e0b7" containerName="main"
Nov 04 04:01:10.390492 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.390472    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.399646 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.399595    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"]
Nov 04 04:01:10.506314 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.506281    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:10.528218 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.528187    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f72hb\" (UniqueName: \"kubernetes.io/projected/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-kube-api-access-f72hb\") pod \"maas-api-revoke-sub-0619452a-6tk5j\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.528369 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.528244    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0619452a-6tk5j\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.528438 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.528398    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0619452a-6tk5j\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.632869 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.630344    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f72hb\" (UniqueName: \"kubernetes.io/projected/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-kube-api-access-f72hb\") pod \"maas-api-revoke-sub-0619452a-6tk5j\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.632869 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.630440    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0619452a-6tk5j\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.632869 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.630502    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0619452a-6tk5j\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.632869 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.631409    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0619452a-6tk5j\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.651413 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.641517    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0619452a-6tk5j\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.652657 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.651607    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f72hb\" (UniqueName: \"kubernetes.io/projected/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-kube-api-access-f72hb\") pod \"maas-api-revoke-sub-0619452a-6tk5j\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.704419 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.704390    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:10.871970 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.871940    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:10.894994 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.894969    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:10.996442 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:10.996408    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"]
Nov 04 04:01:11.001249 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:11.001216    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d4f2ef_c1eb_4212_9c39_2dfe538d4fbc.slice/crio-be21e003310f92c18b303932d6feb33f3964078454aed5b543ecdb1c7b57e7ca WatchSource:0}: Error finding container be21e003310f92c18b303932d6feb33f3964078454aed5b543ecdb1c7b57e7ca: Status 404 returned error can't find the container with id be21e003310f92c18b303932d6feb33f3964078454aed5b543ecdb1c7b57e7ca
Nov 04 04:01:11.125582 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.125542    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:11.138632 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.138561    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:11.241673 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.241637    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19163451-75a0-4fe0-a8c4-97c280b2ebd7-kube-api-access-crqcg\" (UniqueName: \"kubernetes.io/projected/19163451-75a0-4fe0-a8c4-97c280b2ebd7-kube-api-access-crqcg\") pod \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") "
Nov 04 04:01:11.241853 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.241683    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-openshift-ca\") pod \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") "
Nov 04 04:01:11.241853 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.241821    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-xks-ca\") pod \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\" (UID: \"19163451-75a0-4fe0-a8c4-97c280b2ebd7\") "
Nov 04 04:01:11.242099 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.242071    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-openshift-ca" pod "19163451-75a0-4fe0-a8c4-97c280b2ebd7" (UID: "19163451-75a0-4fe0-a8c4-97c280b2ebd7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:11.242196 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.242150    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.243876 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.243851    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-xks-ca" pod "19163451-75a0-4fe0-a8c4-97c280b2ebd7" (UID: "19163451-75a0-4fe0-a8c4-97c280b2ebd7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:11.243968 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.243914    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19163451-75a0-4fe0-a8c4-97c280b2ebd7-kube-api-access-crqcg" pod "19163451-75a0-4fe0-a8c4-97c280b2ebd7" (UID: "19163451-75a0-4fe0-a8c4-97c280b2ebd7"). InnerVolumeSpecName "kube-api-access-crqcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:11.343032 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.343008    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crqcg\" (UniqueName: \"kubernetes.io/projected/19163451-75a0-4fe0-a8c4-97c280b2ebd7-kube-api-access-crqcg\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.343032 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.343031    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19163451-75a0-4fe0-a8c4-97c280b2ebd7-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.432167 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.432092    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw" podUID="b457c8a5-fc7a-426d-807e-68e90ee6d97c" containerName="main" containerID="cri-o://b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b" gracePeriod=45
Nov 04 04:01:11.432806 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.432776    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs" podUID="59151ae2-ff43-45f0-a6bf-b5578532ce57" containerName="main" containerID="cri-o://b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd" gracePeriod=45
Nov 04 04:01:11.546465 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.546426    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:11.708846 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.708800    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"]
Nov 04 04:01:11.713210 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.713183    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 04:01:11.753541 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.753508    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 04:01:11.851793 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.851764    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-tmp-dir\") pod \"59151ae2-ff43-45f0-a6bf-b5578532ce57\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") "
Nov 04 04:01:11.851793 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.851800    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59151ae2-ff43-45f0-a6bf-b5578532ce57-tls-certs\" (UniqueName: \"kubernetes.io/secret/59151ae2-ff43-45f0-a6bf-b5578532ce57-tls-certs\") pod \"59151ae2-ff43-45f0-a6bf-b5578532ce57\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") "
Nov 04 04:01:11.852020 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.851840    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-model-cache\") pod \"59151ae2-ff43-45f0-a6bf-b5578532ce57\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") "
Nov 04 04:01:11.852020 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.851895    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-dshm\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-dshm\") pod \"59151ae2-ff43-45f0-a6bf-b5578532ce57\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") "
Nov 04 04:01:11.852020 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.851944    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-dshm\") pod \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") "
Nov 04 04:01:11.852020 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.851972    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-home\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-home\") pod \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") "
Nov 04 04:01:11.852020 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.852011    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59151ae2-ff43-45f0-a6bf-b5578532ce57-kube-api-access-lllsk\" (UniqueName: \"kubernetes.io/projected/59151ae2-ff43-45f0-a6bf-b5578532ce57-kube-api-access-lllsk\") pod \"59151ae2-ff43-45f0-a6bf-b5578532ce57\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") "
Nov 04 04:01:11.852272 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.852020    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-tmp-dir" pod "59151ae2-ff43-45f0-a6bf-b5578532ce57" (UID: "59151ae2-ff43-45f0-a6bf-b5578532ce57"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:11.852272 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.852045    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b457c8a5-fc7a-426d-807e-68e90ee6d97c-kube-api-access-zsczq\" (UniqueName: \"kubernetes.io/projected/b457c8a5-fc7a-426d-807e-68e90ee6d97c-kube-api-access-zsczq\") pod \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") "
Nov 04 04:01:11.852272 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.852074    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tls-certs\" (UniqueName: \"kubernetes.io/secret/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tls-certs\") pod \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") "
Nov 04 04:01:11.852272 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.852140    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-home\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-home\") pod \"59151ae2-ff43-45f0-a6bf-b5578532ce57\" (UID: \"59151ae2-ff43-45f0-a6bf-b5578532ce57\") "
Nov 04 04:01:11.852272 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.852200    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-model-cache\") pod \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") "
Nov 04 04:01:11.852272 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.852248    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-home" pod "b457c8a5-fc7a-426d-807e-68e90ee6d97c" (UID: "b457c8a5-fc7a-426d-807e-68e90ee6d97c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:11.852593 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.852312    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tmp-dir\") pod \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\" (UID: \"b457c8a5-fc7a-426d-807e-68e90ee6d97c\") "
Nov 04 04:01:11.852990 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.852712    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-model-cache" pod "b457c8a5-fc7a-426d-807e-68e90ee6d97c" (UID: "b457c8a5-fc7a-426d-807e-68e90ee6d97c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:11.852990 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.852970    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tmp-dir" pod "b457c8a5-fc7a-426d-807e-68e90ee6d97c" (UID: "b457c8a5-fc7a-426d-807e-68e90ee6d97c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:11.853191 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.853173    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-model-cache" pod "59151ae2-ff43-45f0-a6bf-b5578532ce57" (UID: "59151ae2-ff43-45f0-a6bf-b5578532ce57"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:11.853397 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.853362    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-home" pod "59151ae2-ff43-45f0-a6bf-b5578532ce57" (UID: "59151ae2-ff43-45f0-a6bf-b5578532ce57"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:11.853787 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.853743    6751 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-home\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.853787 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.853776    6751 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-tmp-dir\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.853988 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.853791    6751 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-home\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.854852 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.854821    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tls-certs" pod "b457c8a5-fc7a-426d-807e-68e90ee6d97c" (UID: "b457c8a5-fc7a-426d-807e-68e90ee6d97c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:11.854852 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.854837    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-dshm" pod "b457c8a5-fc7a-426d-807e-68e90ee6d97c" (UID: "b457c8a5-fc7a-426d-807e-68e90ee6d97c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:11.855015 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.854943    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-dshm" pod "59151ae2-ff43-45f0-a6bf-b5578532ce57" (UID: "59151ae2-ff43-45f0-a6bf-b5578532ce57"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:11.855554 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.855531    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59151ae2-ff43-45f0-a6bf-b5578532ce57-kube-api-access-lllsk" pod "59151ae2-ff43-45f0-a6bf-b5578532ce57" (UID: "59151ae2-ff43-45f0-a6bf-b5578532ce57"). InnerVolumeSpecName "kube-api-access-lllsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:11.855856 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.855835    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59151ae2-ff43-45f0-a6bf-b5578532ce57-tls-certs" pod "59151ae2-ff43-45f0-a6bf-b5578532ce57" (UID: "59151ae2-ff43-45f0-a6bf-b5578532ce57"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:11.856037 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.856018    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b457c8a5-fc7a-426d-807e-68e90ee6d97c-kube-api-access-zsczq" pod "b457c8a5-fc7a-426d-807e-68e90ee6d97c" (UID: "b457c8a5-fc7a-426d-807e-68e90ee6d97c"). InnerVolumeSpecName "kube-api-access-zsczq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:11.889968 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.889695    6751 generic.go:356] "Generic (PLEG): container finished" podID="b457c8a5-fc7a-426d-807e-68e90ee6d97c" containerID="b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b" exitCode=0
Nov 04 04:01:11.889968 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.889758    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"
Nov 04 04:01:11.889968 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.889859    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw" event={"ID":"b457c8a5-fc7a-426d-807e-68e90ee6d97c","Type":"ContainerDied","Data":"b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b"}
Nov 04 04:01:11.889968 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.889893    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw" event={"ID":"b457c8a5-fc7a-426d-807e-68e90ee6d97c","Type":"ContainerDied","Data":"06869c054e135a3823d3ba5ad09f52e6b4ea02718df2857f1b04cce76bc974df"}
Nov 04 04:01:11.889968 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.889913    6751 scope.go:122] "RemoveContainer" containerID="b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b"
Nov 04 04:01:11.892270 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.892242    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j" event={"ID":"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc","Type":"ContainerStarted","Data":"d6a659121d836b8dc6dabe3b35d87bcec6c0016e142419885a2a2a832b337304"}
Nov 04 04:01:11.892363 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.892294    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j" event={"ID":"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc","Type":"ContainerStarted","Data":"be21e003310f92c18b303932d6feb33f3964078454aed5b543ecdb1c7b57e7ca"}
Nov 04 04:01:11.895025 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.894994    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg" event={"ID":"19163451-75a0-4fe0-a8c4-97c280b2ebd7","Type":"ContainerDied","Data":"05d60a0ec3633cb6926b4a0fe83fdf018160b97754095a5a7f5d067c10384942"}
Nov 04 04:01:11.895136 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.895042    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05d60a0ec3633cb6926b4a0fe83fdf018160b97754095a5a7f5d067c10384942"
Nov 04 04:01:11.895136 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.895114    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d1017559-g47dg"
Nov 04 04:01:11.895997 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.895975    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:11.898224 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.898199    6751 generic.go:356] "Generic (PLEG): container finished" podID="59151ae2-ff43-45f0-a6bf-b5578532ce57" containerID="b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd" exitCode=0
Nov 04 04:01:11.898325 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.898261    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs" event={"ID":"59151ae2-ff43-45f0-a6bf-b5578532ce57","Type":"ContainerDied","Data":"b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd"}
Nov 04 04:01:11.898325 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.898283    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"
Nov 04 04:01:11.898325 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.898296    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs" event={"ID":"59151ae2-ff43-45f0-a6bf-b5578532ce57","Type":"ContainerDied","Data":"129747529f008d3412182f273a53f0338366c14f8937767e7ed4252bf8854bdf"}
Nov 04 04:01:11.901647 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.901633    6751 scope.go:122] "RemoveContainer" containerID="b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b"
Nov 04 04:01:11.901979 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:11.901951    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b\": container with ID starting with b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b not found: ID does not exist" containerID="b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b"
Nov 04 04:01:11.902084 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.901990    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b"} err="failed to get container status \"b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b\": rpc error: code = NotFound desc = could not find container \"b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b\": container with ID starting with b126428f985db5dbdb51c807d3776f7939f937f39206495a5f75a17625ec2e7b not found: ID does not exist"
Nov 04 04:01:11.902084 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.902014    6751 scope.go:122] "RemoveContainer" containerID="b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd"
Nov 04 04:01:11.911771 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.911753    6751 scope.go:122] "RemoveContainer" containerID="b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd"
Nov 04 04:01:11.912033 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:11.912015    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd\": container with ID starting with b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd not found: ID does not exist" containerID="b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd"
Nov 04 04:01:11.912088 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.912038    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd"} err="failed to get container status \"b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd\": rpc error: code = NotFound desc = could not find container \"b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd\": container with ID starting with b65437725fb932fda70e68cda25fd29d184882ad14d3c12add3d0aeb726f97dd not found: ID does not exist"
Nov 04 04:01:11.919022 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.918986    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j" podStartSLOduration=1.918974642 podStartE2EDuration="1.918974642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:11.916996117 +0000 UTC m=+666.250933995" watchObservedRunningTime="2026-11-04 04:01:11.918974642 +0000 UTC m=+666.252912519"
Nov 04 04:01:11.943671 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.943580    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"]
Nov 04 04:01:11.946799 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.946774    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/simulator-w6-6a9cdb-kserve-6964c84688-n8pvs"]
Nov 04 04:01:11.956739 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.955724    6751 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tmp-dir\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.956739 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.955752    6751 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59151ae2-ff43-45f0-a6bf-b5578532ce57-tls-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.956739 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.955765    6751 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-model-cache\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.956739 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.955777    6751 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59151ae2-ff43-45f0-a6bf-b5578532ce57-dshm\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.956739 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.955790    6751 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-dshm\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.956739 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.955804    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lllsk\" (UniqueName: \"kubernetes.io/projected/59151ae2-ff43-45f0-a6bf-b5578532ce57-kube-api-access-lllsk\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.956739 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.955818    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsczq\" (UniqueName: \"kubernetes.io/projected/b457c8a5-fc7a-426d-807e-68e90ee6d97c-kube-api-access-zsczq\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.956739 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.955830    6751 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b457c8a5-fc7a-426d-807e-68e90ee6d97c-tls-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.956739 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.955842    6751 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b457c8a5-fc7a-426d-807e-68e90ee6d97c-model-cache\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:11.970091 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.970059    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"]
Nov 04 04:01:11.974205 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:11.974174    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-6a9cdb/premium-w6-6a9cdb-kserve-764dcbdbb9-mdnmw"]
Nov 04 04:01:12.135338 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.135293    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s" podUID="df952825-2dfe-4016-bf56-dd65d66aa8f1" containerName="payload-pre-processing" containerID="cri-o://990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d" gracePeriod=25
Nov 04 04:01:12.140263 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.140237    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:12.408516 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.408482    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"]
Nov 04 04:01:12.409159 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.409139    6751 cpu_manager.go:414] "containerMap: removing container" podUID="b457c8a5-fc7a-426d-807e-68e90ee6d97c" containerName="main"
Nov 04 04:01:12.409244 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.409162    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b457c8a5-fc7a-426d-807e-68e90ee6d97c" containerName="main"
Nov 04 04:01:12.409244 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.409186    6751 cpu_manager.go:414] "containerMap: removing container" podUID="59151ae2-ff43-45f0-a6bf-b5578532ce57" containerName="main"
Nov 04 04:01:12.409244 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.409193    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59151ae2-ff43-45f0-a6bf-b5578532ce57" containerName="main"
Nov 04 04:01:12.409244 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.409228    6751 cpu_manager.go:414] "containerMap: removing container" podUID="19163451-75a0-4fe0-a8c4-97c280b2ebd7" containerName="revoke-keys"
Nov 04 04:01:12.409244 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.409237    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19163451-75a0-4fe0-a8c4-97c280b2ebd7" containerName="revoke-keys"
Nov 04 04:01:12.409481 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.409352    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="b457c8a5-fc7a-426d-807e-68e90ee6d97c" containerName="main"
Nov 04 04:01:12.409481 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.409367    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="59151ae2-ff43-45f0-a6bf-b5578532ce57" containerName="main"
Nov 04 04:01:12.409481 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.409380    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="19163451-75a0-4fe0-a8c4-97c280b2ebd7" containerName="revoke-keys"
Nov 04 04:01:12.412563 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.412543    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.423697 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.423672    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"]
Nov 04 04:01:12.449513 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.449489    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:12.575903 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.571877    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-28adf67f-rnb9l\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.575903 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.571990    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-xks-ca\") pod \"maas-api-revoke-sub-28adf67f-rnb9l\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.575903 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.572101    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d2xn\" (UniqueName: \"kubernetes.io/projected/31bba333-47fa-431a-b65b-eaf49669c567-kube-api-access-7d2xn\") pod \"maas-api-revoke-sub-28adf67f-rnb9l\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.660526 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.660444    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:12.672785 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.672762    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d2xn\" (UniqueName: \"kubernetes.io/projected/31bba333-47fa-431a-b65b-eaf49669c567-kube-api-access-7d2xn\") pod \"maas-api-revoke-sub-28adf67f-rnb9l\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.672861 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.672847    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-28adf67f-rnb9l\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.672911 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.672901    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-xks-ca\") pod \"maas-api-revoke-sub-28adf67f-rnb9l\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.673483 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.673465    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-28adf67f-rnb9l\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.675231 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.675209    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-xks-ca\") pod \"maas-api-revoke-sub-28adf67f-rnb9l\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.687606 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.687585    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d2xn\" (UniqueName: \"kubernetes.io/projected/31bba333-47fa-431a-b65b-eaf49669c567-kube-api-access-7d2xn\") pod \"maas-api-revoke-sub-28adf67f-rnb9l\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.727862 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.727839    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:12.866314 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.866256    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:12.907167 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.907135    6751 generic.go:356] "Generic (PLEG): container finished" podID="e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc" containerID="d6a659121d836b8dc6dabe3b35d87bcec6c0016e142419885a2a2a832b337304" exitCode=0
Nov 04 04:01:12.907306 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:12.907181    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j" event={"ID":"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc","Type":"ContainerDied","Data":"d6a659121d836b8dc6dabe3b35d87bcec6c0016e142419885a2a2a832b337304"}
Nov 04 04:01:13.004906 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.004883    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"]
Nov 04 04:01:13.006345 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:13.006321    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31bba333_47fa_431a_b65b_eaf49669c567.slice/crio-3456085c4218a4f0d05f200ea29b6401b08e477c2946357e41b48af41604b61f WatchSource:0}: Error finding container 3456085c4218a4f0d05f200ea29b6401b08e477c2946357e41b48af41604b61f: Status 404 returned error can't find the container with id 3456085c4218a4f0d05f200ea29b6401b08e477c2946357e41b48af41604b61f
Nov 04 04:01:13.073582 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.073558    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:13.314217 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.314186    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s" podUID="df952825-2dfe-4016-bf56-dd65d66aa8f1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 04:01:13.642429 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.642399    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:13.785494 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.785464    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59151ae2-ff43-45f0-a6bf-b5578532ce57" path="/var/lib/kubelet/pods/59151ae2-ff43-45f0-a6bf-b5578532ce57/volumes"
Nov 04 04:01:13.785887 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.785872    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b457c8a5-fc7a-426d-807e-68e90ee6d97c" path="/var/lib/kubelet/pods/b457c8a5-fc7a-426d-807e-68e90ee6d97c/volumes"
Nov 04 04:01:13.788026 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.787994    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc" podUID="0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" containerName="main" containerID="cri-o://1f9b57142ffd9b626b74f658a8b9656779854fc8208b92737562946756898a9a" gracePeriod=45
Nov 04 04:01:13.794706 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.794498    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9" podUID="8e7f9bbb-441c-41cf-83c4-ae7d565b690b" containerName="main" containerID="cri-o://bde4b37f2a44ed03f3ed9e8e0cbfc4b114936f9aef17858551f7cfefae4a17ee" gracePeriod=45
Nov 04 04:01:13.915931 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.915902    6751 generic.go:356] "Generic (PLEG): container finished" podID="0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" containerID="1f9b57142ffd9b626b74f658a8b9656779854fc8208b92737562946756898a9a" exitCode=0
Nov 04 04:01:13.916088 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.916000    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc" event={"ID":"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4","Type":"ContainerDied","Data":"1f9b57142ffd9b626b74f658a8b9656779854fc8208b92737562946756898a9a"}
Nov 04 04:01:13.917967 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.917938    6751 generic.go:356] "Generic (PLEG): container finished" podID="31bba333-47fa-431a-b65b-eaf49669c567" containerID="c63c909a780c61dbeaed0a58c334394173fea6c82ca5005e088062c69863eed2" exitCode=0
Nov 04 04:01:13.918106 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.918084    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l" event={"ID":"31bba333-47fa-431a-b65b-eaf49669c567","Type":"ContainerDied","Data":"c63c909a780c61dbeaed0a58c334394173fea6c82ca5005e088062c69863eed2"}
Nov 04 04:01:13.918175 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.918144    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l" event={"ID":"31bba333-47fa-431a-b65b-eaf49669c567","Type":"ContainerStarted","Data":"3456085c4218a4f0d05f200ea29b6401b08e477c2946357e41b48af41604b61f"}
Nov 04 04:01:13.920001 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.919979    6751 generic.go:356] "Generic (PLEG): container finished" podID="8e7f9bbb-441c-41cf-83c4-ae7d565b690b" containerID="bde4b37f2a44ed03f3ed9e8e0cbfc4b114936f9aef17858551f7cfefae4a17ee" exitCode=0
Nov 04 04:01:13.920109 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.920042    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9" event={"ID":"8e7f9bbb-441c-41cf-83c4-ae7d565b690b","Type":"ContainerDied","Data":"bde4b37f2a44ed03f3ed9e8e0cbfc4b114936f9aef17858551f7cfefae4a17ee"}
Nov 04 04:01:13.958642 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:13.956989    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9" podUID="8e7f9bbb-441c-41cf-83c4-ae7d565b690b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": dial tcp 10.134.0.46:8000: connect: connection refused"
Nov 04 04:01:14.082722 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.082701    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 04:01:14.086350 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.086325    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 04:01:14.090292 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.090269    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:14.187038 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.186970    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-model-cache\") pod \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") "
Nov 04 04:01:14.187038 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187006    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-model-cache\") pod \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") "
Nov 04 04:01:14.187038 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187033    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-xks-ca\") pod \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") "
Nov 04 04:01:14.187304 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187094    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-home\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-home\") pod \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") "
Nov 04 04:01:14.187304 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187138    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-kube-api-access-f72hb\" (UniqueName: \"kubernetes.io/projected/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-kube-api-access-f72hb\") pod \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") "
Nov 04 04:01:14.187304 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187175    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-dshm\") pod \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") "
Nov 04 04:01:14.187304 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187208    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-kube-api-access-z98wv\" (UniqueName: \"kubernetes.io/projected/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-kube-api-access-z98wv\") pod \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") "
Nov 04 04:01:14.187304 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187265    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-openshift-ca\") pod \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\" (UID: \"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc\") "
Nov 04 04:01:14.187563 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187288    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-model-cache" pod "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" (UID: "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:14.187563 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187318    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-kube-api-access-h8dgp\" (UniqueName: \"kubernetes.io/projected/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-kube-api-access-h8dgp\") pod \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") "
Nov 04 04:01:14.187563 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187356    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-home" pod "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" (UID: "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:14.187563 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187415    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tls-certs\" (UniqueName: \"kubernetes.io/secret/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tls-certs\") pod \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") "
Nov 04 04:01:14.187563 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187450    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tmp-dir\") pod \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") "
Nov 04 04:01:14.187563 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187499    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-dshm\") pod \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\" (UID: \"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4\") "
Nov 04 04:01:14.187563 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187561    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tls-certs\" (UniqueName: \"kubernetes.io/secret/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tls-certs\") pod \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") "
Nov 04 04:01:14.187966 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187649    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tmp-dir\") pod \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") "
Nov 04 04:01:14.187966 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187667    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-openshift-ca" pod "e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc" (UID: "e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:14.187966 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187700    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-home\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-home\") pod \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\" (UID: \"8e7f9bbb-441c-41cf-83c4-ae7d565b690b\") "
Nov 04 04:01:14.187966 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.187887    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tmp-dir" pod "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" (UID: "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:14.188177 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.188134    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-model-cache" pod "8e7f9bbb-441c-41cf-83c4-ae7d565b690b" (UID: "8e7f9bbb-441c-41cf-83c4-ae7d565b690b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:14.188256 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.188232    6751 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tmp-dir\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.188308 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.188259    6751 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-model-cache\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.188308 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.188277    6751 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-model-cache\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.188308 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.188292    6751 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-home\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.188463 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.188308    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.188535 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.188514    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tmp-dir" pod "8e7f9bbb-441c-41cf-83c4-ae7d565b690b" (UID: "8e7f9bbb-441c-41cf-83c4-ae7d565b690b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:14.188755 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.188732    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-home" pod "8e7f9bbb-441c-41cf-83c4-ae7d565b690b" (UID: "8e7f9bbb-441c-41cf-83c4-ae7d565b690b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:14.189722 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.189693    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-xks-ca" pod "e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc" (UID: "e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:14.190046 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.190020    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-kube-api-access-z98wv" pod "8e7f9bbb-441c-41cf-83c4-ae7d565b690b" (UID: "8e7f9bbb-441c-41cf-83c4-ae7d565b690b"). InnerVolumeSpecName "kube-api-access-z98wv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:14.190464 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.190444    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-dshm" pod "8e7f9bbb-441c-41cf-83c4-ae7d565b690b" (UID: "8e7f9bbb-441c-41cf-83c4-ae7d565b690b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:14.190464 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.190460    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-kube-api-access-h8dgp" pod "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" (UID: "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4"). InnerVolumeSpecName "kube-api-access-h8dgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:14.190591 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.190445    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-kube-api-access-f72hb" pod "e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc" (UID: "e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc"). InnerVolumeSpecName "kube-api-access-f72hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:14.190659 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.190590    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tls-certs" pod "8e7f9bbb-441c-41cf-83c4-ae7d565b690b" (UID: "8e7f9bbb-441c-41cf-83c4-ae7d565b690b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:14.190703 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.190687    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tls-certs" pod "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" (UID: "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:14.191005 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.190985    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-dshm" pod "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" (UID: "0afc69ba-a227-4c0a-8cd4-323e7da7f9b4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:14.289548 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.289519    6751 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-dshm\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.289548 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.289544    6751 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tls-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.289548 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.289553    6751 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-tmp-dir\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.289779 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.289561    6751 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-home\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.289779 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.289571    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.289779 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.289580    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f72hb\" (UniqueName: \"kubernetes.io/projected/e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc-kube-api-access-f72hb\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.289779 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.289588    6751 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-dshm\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.289779 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.289597    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z98wv\" (UniqueName: \"kubernetes.io/projected/8e7f9bbb-441c-41cf-83c4-ae7d565b690b-kube-api-access-z98wv\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.289779 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.289605    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8dgp\" (UniqueName: \"kubernetes.io/projected/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-kube-api-access-h8dgp\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.289779 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.289615    6751 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4-tls-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:14.554545 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.554464    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:14.926834 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.926807    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"
Nov 04 04:01:14.927273 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.926854    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9" event={"ID":"8e7f9bbb-441c-41cf-83c4-ae7d565b690b","Type":"ContainerDied","Data":"39cc2e73d28228577c241664cb6363bc204adf04395eff5cc850802a5eb8970e"}
Nov 04 04:01:14.927273 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.926908    6751 scope.go:122] "RemoveContainer" containerID="bde4b37f2a44ed03f3ed9e8e0cbfc4b114936f9aef17858551f7cfefae4a17ee"
Nov 04 04:01:14.928275 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.928198    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j"
Nov 04 04:01:14.928275 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.928213    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0619452a-6tk5j" event={"ID":"e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc","Type":"ContainerDied","Data":"be21e003310f92c18b303932d6feb33f3964078454aed5b543ecdb1c7b57e7ca"}
Nov 04 04:01:14.928275 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.928258    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be21e003310f92c18b303932d6feb33f3964078454aed5b543ecdb1c7b57e7ca"
Nov 04 04:01:14.929820 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.929780    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"
Nov 04 04:01:14.929820 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.929808    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc" event={"ID":"0afc69ba-a227-4c0a-8cd4-323e7da7f9b4","Type":"ContainerDied","Data":"1dd3ef1c5821110d7665528e43dc71d40d0d77f9a38743c04f4e6afd2d0d19f8"}
Nov 04 04:01:14.937049 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.937020    6751 scope.go:122] "RemoveContainer" containerID="1f9b57142ffd9b626b74f658a8b9656779854fc8208b92737562946756898a9a"
Nov 04 04:01:14.958219 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.958185    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"]
Nov 04 04:01:14.961800 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.961772    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-36730f/simulator-w1-36730f-kserve-7fdc4856c9-vpnm9"]
Nov 04 04:01:14.988685 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.988654    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"]
Nov 04 04:01:14.995002 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:14.994974    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-36730f/premium-w1-36730f-kserve-756c8bb874-h77mc"]
Nov 04 04:01:15.085662 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.085639    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:15.198956 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.198882    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31bba333-47fa-431a-b65b-eaf49669c567-kube-api-access-7d2xn\" (UniqueName: \"kubernetes.io/projected/31bba333-47fa-431a-b65b-eaf49669c567-kube-api-access-7d2xn\") pod \"31bba333-47fa-431a-b65b-eaf49669c567\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") "
Nov 04 04:01:15.199098 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.198965    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-openshift-ca\") pod \"31bba333-47fa-431a-b65b-eaf49669c567\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") "
Nov 04 04:01:15.199098 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.199005    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-xks-ca\") pod \"31bba333-47fa-431a-b65b-eaf49669c567\" (UID: \"31bba333-47fa-431a-b65b-eaf49669c567\") "
Nov 04 04:01:15.199347 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.199322    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-openshift-ca" pod "31bba333-47fa-431a-b65b-eaf49669c567" (UID: "31bba333-47fa-431a-b65b-eaf49669c567"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:15.201081 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.201053    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31bba333-47fa-431a-b65b-eaf49669c567-kube-api-access-7d2xn" pod "31bba333-47fa-431a-b65b-eaf49669c567" (UID: "31bba333-47fa-431a-b65b-eaf49669c567"). InnerVolumeSpecName "kube-api-access-7d2xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:15.201182 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.201085    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-xks-ca" pod "31bba333-47fa-431a-b65b-eaf49669c567" (UID: "31bba333-47fa-431a-b65b-eaf49669c567"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:15.298418 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.298381    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:15.300824 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.300805    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:15.300882 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.300828    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31bba333-47fa-431a-b65b-eaf49669c567-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:15.300882 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.300840    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d2xn\" (UniqueName: \"kubernetes.io/projected/31bba333-47fa-431a-b65b-eaf49669c567-kube-api-access-7d2xn\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:15.610968 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.610451    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:15.785601 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.785568    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" path="/var/lib/kubelet/pods/0afc69ba-a227-4c0a-8cd4-323e7da7f9b4/volumes"
Nov 04 04:01:15.786049 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.786033    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e7f9bbb-441c-41cf-83c4-ae7d565b690b" path="/var/lib/kubelet/pods/8e7f9bbb-441c-41cf-83c4-ae7d565b690b/volumes"
Nov 04 04:01:15.847588 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.847558    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:15.937575 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.937486    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l"
Nov 04 04:01:15.937575 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.937513    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28adf67f-rnb9l" event={"ID":"31bba333-47fa-431a-b65b-eaf49669c567","Type":"ContainerDied","Data":"3456085c4218a4f0d05f200ea29b6401b08e477c2946357e41b48af41604b61f"}
Nov 04 04:01:15.938104 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:15.937591    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3456085c4218a4f0d05f200ea29b6401b08e477c2946357e41b48af41604b61f"
Nov 04 04:01:16.407387 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407354    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"]
Nov 04 04:01:16.407816 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407802    6751 cpu_manager.go:414] "containerMap: removing container" podUID="e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc" containerName="revoke-keys"
Nov 04 04:01:16.407862 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407817    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc" containerName="revoke-keys"
Nov 04 04:01:16.407862 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407836    6751 cpu_manager.go:414] "containerMap: removing container" podUID="0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" containerName="main"
Nov 04 04:01:16.407862 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407843    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" containerName="main"
Nov 04 04:01:16.407862 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407853    6751 cpu_manager.go:414] "containerMap: removing container" podUID="8e7f9bbb-441c-41cf-83c4-ae7d565b690b" containerName="main"
Nov 04 04:01:16.407862 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407858    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e7f9bbb-441c-41cf-83c4-ae7d565b690b" containerName="main"
Nov 04 04:01:16.408016 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407872    6751 cpu_manager.go:414] "containerMap: removing container" podUID="31bba333-47fa-431a-b65b-eaf49669c567" containerName="revoke-keys"
Nov 04 04:01:16.408016 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407876    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31bba333-47fa-431a-b65b-eaf49669c567" containerName="revoke-keys"
Nov 04 04:01:16.408016 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407936    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="0afc69ba-a227-4c0a-8cd4-323e7da7f9b4" containerName="main"
Nov 04 04:01:16.408016 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407945    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e7f9bbb-441c-41cf-83c4-ae7d565b690b" containerName="main"
Nov 04 04:01:16.408016 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407952    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="31bba333-47fa-431a-b65b-eaf49669c567" containerName="revoke-keys"
Nov 04 04:01:16.408016 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.407961    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc" containerName="revoke-keys"
Nov 04 04:01:16.411143 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.411127    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.418887 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.418859    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4afe0e\"/\"distinct-2-w0-4afe0e-kserve-self-signed-certs\""
Nov 04 04:01:16.419047 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.418997    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4afe0e\"/\"default-dockercfg-gqkhp\""
Nov 04 04:01:16.420766 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.420742    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4afe0e\"/\"openshift-service-ca.crt\""
Nov 04 04:01:16.420914 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.420884    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4afe0e\"/\"kube-root-ca.crt\""
Nov 04 04:01:16.422141 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.422113    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:16.422833 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.422813    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"]
Nov 04 04:01:16.518171 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.518141    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-model-cache\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.518328 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.518177    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-dshm\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.518328 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.518202    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1069831-769d-4c65-b634-f6c12d2e00b9-tls-certs\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.518393 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.518342    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-home\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.518473 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.518453    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmcb6\" (UniqueName: \"kubernetes.io/projected/e1069831-769d-4c65-b634-f6c12d2e00b9-kube-api-access-qmcb6\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.518529 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.518508    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-tmp-dir\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.620021 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.619989    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-tmp-dir\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.620182 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.620058    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-model-cache\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.620240 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.620216    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-dshm\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.620318 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.620268    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1069831-769d-4c65-b634-f6c12d2e00b9-tls-certs\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.620371 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.620339    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-home\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.620424 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.620406    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-model-cache\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.620489 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.620472    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmcb6\" (UniqueName: \"kubernetes.io/projected/e1069831-769d-4c65-b634-f6c12d2e00b9-kube-api-access-qmcb6\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.620489 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.620480    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-tmp-dir\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.620709 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.620691    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-home\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.622402 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.622380    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-dshm\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.622787 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.622771    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1069831-769d-4c65-b634-f6c12d2e00b9-tls-certs\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.632932 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.632913    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmcb6\" (UniqueName: \"kubernetes.io/projected/e1069831-769d-4c65-b634-f6c12d2e00b9-kube-api-access-qmcb6\") pod \"distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") " pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.646335 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.646315    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:16.709268 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.709194    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d" podUID="f6916ba0-d2af-4ee9-8829-7d68eec86456" containerName="payload-processing" containerID="cri-o://69d5554219216821498da520b7f9080138213e36115391961a7491984ec9cede" gracePeriod=25
Nov 04 04:01:16.726936 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.726918    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:16.947228 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.947200    6751 generic.go:356] "Generic (PLEG): container finished" podID="f6916ba0-d2af-4ee9-8829-7d68eec86456" containerID="69d5554219216821498da520b7f9080138213e36115391961a7491984ec9cede" exitCode=0
Nov 04 04:01:16.947594 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.947281    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d" event={"ID":"f6916ba0-d2af-4ee9-8829-7d68eec86456","Type":"ContainerDied","Data":"69d5554219216821498da520b7f9080138213e36115391961a7491984ec9cede"}
Nov 04 04:01:16.953100 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:16.953079    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 04:01:17.003355 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.003290    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"]
Nov 04 04:01:17.007431 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:17.007406    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1069831_769d_4c65_b634_f6c12d2e00b9.slice/crio-6e8a94dd0c107d54a9dfe065f8160b55d9475a3591a6e9d65d1ddc0e9cccac61 WatchSource:0}: Error finding container 6e8a94dd0c107d54a9dfe065f8160b55d9475a3591a6e9d65d1ddc0e9cccac61: Status 404 returned error can't find the container with id 6e8a94dd0c107d54a9dfe065f8160b55d9475a3591a6e9d65d1ddc0e9cccac61
Nov 04 04:01:17.023860 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.023839    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6916ba0-d2af-4ee9-8829-7d68eec86456-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6916ba0-d2af-4ee9-8829-7d68eec86456-plugins-config-volume\") pod \"f6916ba0-d2af-4ee9-8829-7d68eec86456\" (UID: \"f6916ba0-d2af-4ee9-8829-7d68eec86456\") "
Nov 04 04:01:17.024386 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.024284    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6916ba0-d2af-4ee9-8829-7d68eec86456-plugins-config-volume" pod "f6916ba0-d2af-4ee9-8829-7d68eec86456" (UID: "f6916ba0-d2af-4ee9-8829-7d68eec86456"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:17.024478 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.024449    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6916ba0-d2af-4ee9-8829-7d68eec86456-kube-api-access-vjn65\" (UniqueName: \"kubernetes.io/projected/f6916ba0-d2af-4ee9-8829-7d68eec86456-kube-api-access-vjn65\") pod \"f6916ba0-d2af-4ee9-8829-7d68eec86456\" (UID: \"f6916ba0-d2af-4ee9-8829-7d68eec86456\") "
Nov 04 04:01:17.024756 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.024741    6751 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6916ba0-d2af-4ee9-8829-7d68eec86456-plugins-config-volume\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:17.026877 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.026854    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6916ba0-d2af-4ee9-8829-7d68eec86456-kube-api-access-vjn65" pod "f6916ba0-d2af-4ee9-8829-7d68eec86456" (UID: "f6916ba0-d2af-4ee9-8829-7d68eec86456"). InnerVolumeSpecName "kube-api-access-vjn65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:17.125684 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.125652    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjn65\" (UniqueName: \"kubernetes.io/projected/f6916ba0-d2af-4ee9-8829-7d68eec86456-kube-api-access-vjn65\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:17.274959 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.274878    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:17.955931 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.955857    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75" event={"ID":"e1069831-769d-4c65-b634-f6c12d2e00b9","Type":"ContainerStarted","Data":"a5cc82d72910ca5ab7a01a6220f76a11bce140d6899a6604dbede9e4eb98fdd7"}
Nov 04 04:01:17.955931 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.955903    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75" event={"ID":"e1069831-769d-4c65-b634-f6c12d2e00b9","Type":"ContainerStarted","Data":"6e8a94dd0c107d54a9dfe065f8160b55d9475a3591a6e9d65d1ddc0e9cccac61"}
Nov 04 04:01:17.957247 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.957222    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d" event={"ID":"f6916ba0-d2af-4ee9-8829-7d68eec86456","Type":"ContainerDied","Data":"0681272e814090c614e8cc4273d95a199d3071f147f209b369bd1e9b36e12523"}
Nov 04 04:01:17.957365 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.957266    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"
Nov 04 04:01:17.957365 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.957281    6751 scope.go:122] "RemoveContainer" containerID="69d5554219216821498da520b7f9080138213e36115391961a7491984ec9cede"
Nov 04 04:01:17.977576 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.977533    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75" podStartSLOduration=1.9775206619999999 podStartE2EDuration="1.977520662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:17.975336378 +0000 UTC m=+672.309274256" watchObservedRunningTime="2026-11-04 04:01:17.977520662 +0000 UTC m=+672.311458540"
Nov 04 04:01:17.993259 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.993234    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"]
Nov 04 04:01:17.997923 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:17.997843    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:18.001036 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:18.001015    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-78ca37-5dd495d496-cv49d"]
Nov 04 04:01:18.687256 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:18.687210    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:18.727457 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:18.727420    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:19.060791 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:19.060699    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:19.532043 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:19.532006    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:19.742224 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:19.742078    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:19.785573 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:19.785497    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6916ba0-d2af-4ee9-8829-7d68eec86456" path="/var/lib/kubelet/pods/f6916ba0-d2af-4ee9-8829-7d68eec86456/volumes"
Nov 04 04:01:20.394650 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:20.394589    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:20.615580 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:20.615537    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:21.088287 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:21.088253    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 04:01:21.197856 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.197827    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:21.404142 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.404104    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:21.503776 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.503732    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"]
Nov 04 04:01:21.504296 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.504276    6751 cpu_manager.go:414] "containerMap: removing container" podUID="f6916ba0-d2af-4ee9-8829-7d68eec86456" containerName="payload-processing"
Nov 04 04:01:21.504408 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.504297    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6916ba0-d2af-4ee9-8829-7d68eec86456" containerName="payload-processing"
Nov 04 04:01:21.504477 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.504462    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6916ba0-d2af-4ee9-8829-7d68eec86456" containerName="payload-processing"
Nov 04 04:01:21.512902 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.512880    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:21.517005 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.516973    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 04:01:21.517786 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.517767    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 04:01:21.518002 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.517983    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"]
Nov 04 04:01:21.561453 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.561429    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2wnx7\""
Nov 04 04:01:21.561587 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.561576    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 04:01:21.590950 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.590925    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8f335aa5-4x5dt\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:21.591077 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.590967    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5cw4\" (UniqueName: \"kubernetes.io/projected/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-kube-api-access-n5cw4\") pod \"maas-api-revoke-sub-8f335aa5-4x5dt\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:21.591077 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.590989    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8f335aa5-4x5dt\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:21.692627 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.692551    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8f335aa5-4x5dt\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:21.692627 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.692592    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5cw4\" (UniqueName: \"kubernetes.io/projected/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-kube-api-access-n5cw4\") pod \"maas-api-revoke-sub-8f335aa5-4x5dt\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:21.692818 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.692650    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8f335aa5-4x5dt\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:21.693320 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.693297    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8f335aa5-4x5dt\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:21.694830 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.694808    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8f335aa5-4x5dt\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:21.705869 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.705847    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5cw4\" (UniqueName: \"kubernetes.io/projected/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-kube-api-access-n5cw4\") pod \"maas-api-revoke-sub-8f335aa5-4x5dt\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:21.832232 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:21.832202    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:22.011656 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:22.011522    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:22.111374 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:22.111341    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"]
Nov 04 04:01:22.112880 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:22.112851    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaaf9a0f_0904_4722_80b4_a36d7bd4192d.slice/crio-967b1dc962558f62af8ca9b209780672b95443bc40814cbd2967f385e642696e WatchSource:0}: Error finding container 967b1dc962558f62af8ca9b209780672b95443bc40814cbd2967f385e642696e: Status 404 returned error can't find the container with id 967b1dc962558f62af8ca9b209780672b95443bc40814cbd2967f385e642696e
Nov 04 04:01:22.599169 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:22.599142    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:22.807788 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:22.807757    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:23.003681 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:23.003642    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt" event={"ID":"eaaf9a0f-0904-4722-80b4-a36d7bd4192d","Type":"ContainerStarted","Data":"2057b223fd90c3e58eb44adb1b5995bb05262c9c7e101b975e6bcdb738672aa2"}
Nov 04 04:01:23.003817 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:23.003697    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt" event={"ID":"eaaf9a0f-0904-4722-80b4-a36d7bd4192d","Type":"ContainerStarted","Data":"967b1dc962558f62af8ca9b209780672b95443bc40814cbd2967f385e642696e"}
Nov 04 04:01:23.026522 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:23.026494    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:23.251726 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:23.251403    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:23.313520 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:23.313481    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s" podUID="df952825-2dfe-4016-bf56-dd65d66aa8f1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 04:01:24.010041 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.010006    6751 generic.go:356] "Generic (PLEG): container finished" podID="eaaf9a0f-0904-4722-80b4-a36d7bd4192d" containerID="2057b223fd90c3e58eb44adb1b5995bb05262c9c7e101b975e6bcdb738672aa2" exitCode=0
Nov 04 04:01:24.010532 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.010052    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt" event={"ID":"eaaf9a0f-0904-4722-80b4-a36d7bd4192d","Type":"ContainerDied","Data":"2057b223fd90c3e58eb44adb1b5995bb05262c9c7e101b975e6bcdb738672aa2"}
Nov 04 04:01:24.127002 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.126968    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:24.184669 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.184646    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:24.221918 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.221891    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-kube-api-access-n5cw4\" (UniqueName: \"kubernetes.io/projected/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-kube-api-access-n5cw4\") pod \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") "
Nov 04 04:01:24.222099 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.222062    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-openshift-ca\") pod \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") "
Nov 04 04:01:24.222178 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.222098    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-xks-ca\") pod \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\" (UID: \"eaaf9a0f-0904-4722-80b4-a36d7bd4192d\") "
Nov 04 04:01:24.222508 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.222475    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-openshift-ca" pod "eaaf9a0f-0904-4722-80b4-a36d7bd4192d" (UID: "eaaf9a0f-0904-4722-80b4-a36d7bd4192d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:24.224084 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.224057    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-kube-api-access-n5cw4" pod "eaaf9a0f-0904-4722-80b4-a36d7bd4192d" (UID: "eaaf9a0f-0904-4722-80b4-a36d7bd4192d"). InnerVolumeSpecName "kube-api-access-n5cw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:24.224211 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.224122    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-xks-ca" pod "eaaf9a0f-0904-4722-80b4-a36d7bd4192d" (UID: "eaaf9a0f-0904-4722-80b4-a36d7bd4192d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:24.323925 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.323889    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:24.323925 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.323919    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:24.323925 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.323930    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5cw4\" (UniqueName: \"kubernetes.io/projected/eaaf9a0f-0904-4722-80b4-a36d7bd4192d-kube-api-access-n5cw4\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:24.398645 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.396351    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:24.671885 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.671847    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"]
Nov 04 04:01:24.672314 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.672300    6751 cpu_manager.go:414] "containerMap: removing container" podUID="eaaf9a0f-0904-4722-80b4-a36d7bd4192d" containerName="revoke-keys"
Nov 04 04:01:24.672314 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.672313    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eaaf9a0f-0904-4722-80b4-a36d7bd4192d" containerName="revoke-keys"
Nov 04 04:01:24.672474 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.672385    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="eaaf9a0f-0904-4722-80b4-a36d7bd4192d" containerName="revoke-keys"
Nov 04 04:01:24.675777 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.675750    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:24.686077 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.686052    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"]
Nov 04 04:01:24.727991 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.727951    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsdbp\" (UniqueName: \"kubernetes.io/projected/63ef4111-7492-4f9a-99aa-c1e66c451488-kube-api-access-bsdbp\") pod \"maas-api-revoke-sub-2a982a8a-znmgm\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:24.728201 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.728080    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2a982a8a-znmgm\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:24.728201 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.728120    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2a982a8a-znmgm\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:24.829350 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.829322    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsdbp\" (UniqueName: \"kubernetes.io/projected/63ef4111-7492-4f9a-99aa-c1e66c451488-kube-api-access-bsdbp\") pod \"maas-api-revoke-sub-2a982a8a-znmgm\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:24.829500 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.829381    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2a982a8a-znmgm\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:24.829500 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.829409    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2a982a8a-znmgm\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:24.830172 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.830149    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2a982a8a-znmgm\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:24.831740 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.831720    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2a982a8a-znmgm\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:24.844421 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.844397    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsdbp\" (UniqueName: \"kubernetes.io/projected/63ef4111-7492-4f9a-99aa-c1e66c451488-kube-api-access-bsdbp\") pod \"maas-api-revoke-sub-2a982a8a-znmgm\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:24.993928 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:24.993851    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:25.016823 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:25.016802    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt"
Nov 04 04:01:25.017181 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:25.016836    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f335aa5-4x5dt" event={"ID":"eaaf9a0f-0904-4722-80b4-a36d7bd4192d","Type":"ContainerDied","Data":"967b1dc962558f62af8ca9b209780672b95443bc40814cbd2967f385e642696e"}
Nov 04 04:01:25.017181 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:25.016896    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="967b1dc962558f62af8ca9b209780672b95443bc40814cbd2967f385e642696e"
Nov 04 04:01:25.140314 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:25.140284    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:25.291209 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:25.291119    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"]
Nov 04 04:01:25.292830 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:25.292788    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ef4111_7492_4f9a_99aa_c1e66c451488.slice/crio-b93cfa3db585b004e526bf1c8991fc9c9376a99910a40f950c79cf56407eb4bb WatchSource:0}: Error finding container b93cfa3db585b004e526bf1c8991fc9c9376a99910a40f950c79cf56407eb4bb: Status 404 returned error can't find the container with id b93cfa3db585b004e526bf1c8991fc9c9376a99910a40f950c79cf56407eb4bb
Nov 04 04:01:26.033727 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:26.033604    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm" event={"ID":"63ef4111-7492-4f9a-99aa-c1e66c451488","Type":"ContainerStarted","Data":"81139627375c64b2cf6e83163cc68979b89256e50f0332517e68ef735db301d2"}
Nov 04 04:01:26.034338 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:26.034293    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm" event={"ID":"63ef4111-7492-4f9a-99aa-c1e66c451488","Type":"ContainerStarted","Data":"b93cfa3db585b004e526bf1c8991fc9c9376a99910a40f950c79cf56407eb4bb"}
Nov 04 04:01:26.074075 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:26.074003    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm" podStartSLOduration=2.073983837 podStartE2EDuration="2.073983837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:26.071988505 +0000 UTC m=+680.405926386" watchObservedRunningTime="2026-11-04 04:01:26.073983837 +0000 UTC m=+680.407921716"
Nov 04 04:01:26.088653 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:26.086602    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"]
Nov 04 04:01:26.307741 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:26.305417    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:26.528419 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:26.528394    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:26.727535 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:26.727501    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:26.740063 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:26.740041    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:26.929904 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:26.929870    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:27.042048 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:27.041962    6751 generic.go:356] "Generic (PLEG): container finished" podID="63ef4111-7492-4f9a-99aa-c1e66c451488" containerID="81139627375c64b2cf6e83163cc68979b89256e50f0332517e68ef735db301d2" exitCode=0
Nov 04 04:01:27.042528 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:27.042047    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm" event={"ID":"63ef4111-7492-4f9a-99aa-c1e66c451488","Type":"ContainerDied","Data":"81139627375c64b2cf6e83163cc68979b89256e50f0332517e68ef735db301d2"}
Nov 04 04:01:27.055936 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:27.055902    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:27.144378 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:27.144346    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:27.447539 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:27.447500    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:28.176606 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.176583    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:28.299751 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.299686    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-xks-ca\") pod \"63ef4111-7492-4f9a-99aa-c1e66c451488\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") "
Nov 04 04:01:28.299751 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.299724    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-openshift-ca\") pod \"63ef4111-7492-4f9a-99aa-c1e66c451488\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") "
Nov 04 04:01:28.299960 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.299772    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63ef4111-7492-4f9a-99aa-c1e66c451488-kube-api-access-bsdbp\" (UniqueName: \"kubernetes.io/projected/63ef4111-7492-4f9a-99aa-c1e66c451488-kube-api-access-bsdbp\") pod \"63ef4111-7492-4f9a-99aa-c1e66c451488\" (UID: \"63ef4111-7492-4f9a-99aa-c1e66c451488\") "
Nov 04 04:01:28.300103 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.300079    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-openshift-ca" pod "63ef4111-7492-4f9a-99aa-c1e66c451488" (UID: "63ef4111-7492-4f9a-99aa-c1e66c451488"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:28.302032 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.302000    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-xks-ca" pod "63ef4111-7492-4f9a-99aa-c1e66c451488" (UID: "63ef4111-7492-4f9a-99aa-c1e66c451488"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:28.302149 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.302128    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63ef4111-7492-4f9a-99aa-c1e66c451488-kube-api-access-bsdbp" pod "63ef4111-7492-4f9a-99aa-c1e66c451488" (UID: "63ef4111-7492-4f9a-99aa-c1e66c451488"). InnerVolumeSpecName "kube-api-access-bsdbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:28.400870 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.400840    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:28.400996 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.400875    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsdbp\" (UniqueName: \"kubernetes.io/projected/63ef4111-7492-4f9a-99aa-c1e66c451488-kube-api-access-bsdbp\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:28.400996 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.400892    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63ef4111-7492-4f9a-99aa-c1e66c451488-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:28.583329 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.583244    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:28.853304 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:28.853271    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:29.055699 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:29.055675    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm"
Nov 04 04:01:29.055841 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:29.055716    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a982a8a-znmgm" event={"ID":"63ef4111-7492-4f9a-99aa-c1e66c451488","Type":"ContainerDied","Data":"b93cfa3db585b004e526bf1c8991fc9c9376a99910a40f950c79cf56407eb4bb"}
Nov 04 04:01:29.055841 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:29.055767    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b93cfa3db585b004e526bf1c8991fc9c9376a99910a40f950c79cf56407eb4bb"
Nov 04 04:01:29.435593 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:29.435546    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"]
Nov 04 04:01:29.457493 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:29.457463    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"]
Nov 04 04:01:29.628507 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:29.628474    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:29.840645 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:29.840551    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:30.055639 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:30.055601    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:30.580112 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:30.580082    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:30.786569 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:30.786539    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:31.103113 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.103058    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:31.362099 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.362002    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:31.608413 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.608380    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:31.869473 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.869443    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:31.912084 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.912052    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"]
Nov 04 04:01:31.912488 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.912473    6751 cpu_manager.go:414] "containerMap: removing container" podUID="63ef4111-7492-4f9a-99aa-c1e66c451488" containerName="revoke-keys"
Nov 04 04:01:31.912488 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.912488    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63ef4111-7492-4f9a-99aa-c1e66c451488" containerName="revoke-keys"
Nov 04 04:01:31.912584 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.912573    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="63ef4111-7492-4f9a-99aa-c1e66c451488" containerName="revoke-keys"
Nov 04 04:01:31.916294 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.916276    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:31.921672 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.921653    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 04:01:31.922772 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.922752    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 04:01:31.925735 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.925708    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"]
Nov 04 04:01:31.986070 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.986037    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdrp4\" (UniqueName: \"kubernetes.io/projected/59ace1c6-1907-424a-a071-5d7fd00aea0d-kube-api-access-tdrp4\") pod \"maas-api-revoke-sub-464e2bd6-fmfxd\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:31.986257 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.986089    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-464e2bd6-fmfxd\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:31.986257 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:31.986238    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-464e2bd6-fmfxd\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:32.012008 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.011985    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 04:01:32.012139 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.012057    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2wnx7\""
Nov 04 04:01:32.086937 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.086910    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdrp4\" (UniqueName: \"kubernetes.io/projected/59ace1c6-1907-424a-a071-5d7fd00aea0d-kube-api-access-tdrp4\") pod \"maas-api-revoke-sub-464e2bd6-fmfxd\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:32.087095 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.086942    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-464e2bd6-fmfxd\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:32.087095 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.087024    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-464e2bd6-fmfxd\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:32.087767 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.087729    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-464e2bd6-fmfxd\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:32.089797 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.089772    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-464e2bd6-fmfxd\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:32.099250 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.099231    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdrp4\" (UniqueName: \"kubernetes.io/projected/59ace1c6-1907-424a-a071-5d7fd00aea0d-kube-api-access-tdrp4\") pod \"maas-api-revoke-sub-464e2bd6-fmfxd\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:32.128284 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.128221    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:32.238642 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.234820    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:32.288727 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.278318    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"]
Nov 04 04:01:32.317832 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.317798    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"]
Nov 04 04:01:32.318011 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.317929    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.343181 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.343151    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:32.399408 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.399375    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ght2k\" (UniqueName: \"kubernetes.io/projected/4381712a-9961-4bc9-8b53-be50182422c5-kube-api-access-ght2k\") pod \"maas-api-revoke-sub-66f93416-k5vfs\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.399518 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.399456    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66f93416-k5vfs\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.399591 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.399569    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66f93416-k5vfs\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.476083 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.476059    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 04:01:32.500219 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.500196    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df952825-2dfe-4016-bf56-dd65d66aa8f1-kube-api-access-2dv5w\" (UniqueName: \"kubernetes.io/projected/df952825-2dfe-4016-bf56-dd65d66aa8f1-kube-api-access-2dv5w\") pod \"df952825-2dfe-4016-bf56-dd65d66aa8f1\" (UID: \"df952825-2dfe-4016-bf56-dd65d66aa8f1\") "
Nov 04 04:01:32.500346 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.500265    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df952825-2dfe-4016-bf56-dd65d66aa8f1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df952825-2dfe-4016-bf56-dd65d66aa8f1-plugins-config-volume\") pod \"df952825-2dfe-4016-bf56-dd65d66aa8f1\" (UID: \"df952825-2dfe-4016-bf56-dd65d66aa8f1\") "
Nov 04 04:01:32.500430 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.500419    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ght2k\" (UniqueName: \"kubernetes.io/projected/4381712a-9961-4bc9-8b53-be50182422c5-kube-api-access-ght2k\") pod \"maas-api-revoke-sub-66f93416-k5vfs\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.500493 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.500460    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66f93416-k5vfs\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.500569 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.500510    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66f93416-k5vfs\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.500719 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.500696    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df952825-2dfe-4016-bf56-dd65d66aa8f1-plugins-config-volume" pod "df952825-2dfe-4016-bf56-dd65d66aa8f1" (UID: "df952825-2dfe-4016-bf56-dd65d66aa8f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:32.501377 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.501352    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66f93416-k5vfs\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.502428 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.502407    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df952825-2dfe-4016-bf56-dd65d66aa8f1-kube-api-access-2dv5w" pod "df952825-2dfe-4016-bf56-dd65d66aa8f1" (UID: "df952825-2dfe-4016-bf56-dd65d66aa8f1"). InnerVolumeSpecName "kube-api-access-2dv5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:32.502775 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.502758    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66f93416-k5vfs\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.512475 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.512451    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ght2k\" (UniqueName: \"kubernetes.io/projected/4381712a-9961-4bc9-8b53-be50182422c5-kube-api-access-ght2k\") pod \"maas-api-revoke-sub-66f93416-k5vfs\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.587379 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.587348    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"]
Nov 04 04:01:32.588750 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:32.588722    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ace1c6_1907_424a_a071_5d7fd00aea0d.slice/crio-40ffa8e5e8cc577447dbaa13818633189a206260339297f1fd84bbd412e08834 WatchSource:0}: Error finding container 40ffa8e5e8cc577447dbaa13818633189a206260339297f1fd84bbd412e08834: Status 404 returned error can't find the container with id 40ffa8e5e8cc577447dbaa13818633189a206260339297f1fd84bbd412e08834
Nov 04 04:01:32.602074 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.602049    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dv5w\" (UniqueName: \"kubernetes.io/projected/df952825-2dfe-4016-bf56-dd65d66aa8f1-kube-api-access-2dv5w\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:32.602218 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.602081    6751 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df952825-2dfe-4016-bf56-dd65d66aa8f1-plugins-config-volume\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:32.640434 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.640412    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:32.725884 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.725855    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:32.920841 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:32.920818    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"]
Nov 04 04:01:32.922116 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:32.922086    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4381712a_9961_4bc9_8b53_be50182422c5.slice/crio-b6db082d70d855ee4a32a0359b1bed5edfc30ef0330b5ab320a3f40c0113623b WatchSource:0}: Error finding container b6db082d70d855ee4a32a0359b1bed5edfc30ef0330b5ab320a3f40c0113623b: Status 404 returned error can't find the container with id b6db082d70d855ee4a32a0359b1bed5edfc30ef0330b5ab320a3f40c0113623b
Nov 04 04:01:33.096674 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.096016    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:33.113200 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.112020    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd" event={"ID":"59ace1c6-1907-424a-a071-5d7fd00aea0d","Type":"ContainerStarted","Data":"322e94b23697d75c2a03f330d28e50d94ca858f9cba0914c5fa6304a5f53c0d8"}
Nov 04 04:01:33.113200 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.112092    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd" event={"ID":"59ace1c6-1907-424a-a071-5d7fd00aea0d","Type":"ContainerStarted","Data":"40ffa8e5e8cc577447dbaa13818633189a206260339297f1fd84bbd412e08834"}
Nov 04 04:01:33.115224 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.115194    6751 generic.go:356] "Generic (PLEG): container finished" podID="df952825-2dfe-4016-bf56-dd65d66aa8f1" containerID="990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d" exitCode=0
Nov 04 04:01:33.115346 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.115288    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s" event={"ID":"df952825-2dfe-4016-bf56-dd65d66aa8f1","Type":"ContainerDied","Data":"990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d"}
Nov 04 04:01:33.115346 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.115335    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s" event={"ID":"df952825-2dfe-4016-bf56-dd65d66aa8f1","Type":"ContainerDied","Data":"ccf115cd91737381e52679f45415d4141f582414757d5f61ffcc0aad72b966fe"}
Nov 04 04:01:33.115427 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.115363    6751 scope.go:122] "RemoveContainer" containerID="990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d"
Nov 04 04:01:33.115520 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.115507    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"
Nov 04 04:01:33.125424 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.125399    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs" event={"ID":"4381712a-9961-4bc9-8b53-be50182422c5","Type":"ContainerStarted","Data":"b6db082d70d855ee4a32a0359b1bed5edfc30ef0330b5ab320a3f40c0113623b"}
Nov 04 04:01:33.142023 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.141974    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd" podStartSLOduration=2.141959394 podStartE2EDuration="2.141959394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:33.136237983 +0000 UTC m=+687.470175863" watchObservedRunningTime="2026-11-04 04:01:33.141959394 +0000 UTC m=+687.475897299"
Nov 04 04:01:33.177580 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.177560    6751 scope.go:122] "RemoveContainer" containerID="990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d"
Nov 04 04:01:33.178535 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:33.178512    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d\": container with ID starting with 990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d not found: ID does not exist" containerID="990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d"
Nov 04 04:01:33.178641 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.178547    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d"} err="failed to get container status \"990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d\": rpc error: code = NotFound desc = could not find container \"990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d\": container with ID starting with 990e31afda83e679d9331f7cc132633ebbc9adebdeb04b8a9de8e930d77d0c3d not found: ID does not exist"
Nov 04 04:01:33.186108 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.186084    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"]
Nov 04 04:01:33.192121 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.192099    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-36730f-6df8c45795-sfr6s"]
Nov 04 04:01:33.781286 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.781210    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:33.787068 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.787036    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df952825-2dfe-4016-bf56-dd65d66aa8f1" path="/var/lib/kubelet/pods/df952825-2dfe-4016-bf56-dd65d66aa8f1/volumes"
Nov 04 04:01:33.988948 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:33.988911    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:34.132500 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:34.132466    6751 generic.go:356] "Generic (PLEG): container finished" podID="59ace1c6-1907-424a-a071-5d7fd00aea0d" containerID="322e94b23697d75c2a03f330d28e50d94ca858f9cba0914c5fa6304a5f53c0d8" exitCode=0
Nov 04 04:01:34.132677 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:34.132553    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd" event={"ID":"59ace1c6-1907-424a-a071-5d7fd00aea0d","Type":"ContainerDied","Data":"322e94b23697d75c2a03f330d28e50d94ca858f9cba0914c5fa6304a5f53c0d8"}
Nov 04 04:01:34.135039 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:34.135016    6751 generic.go:356] "Generic (PLEG): container finished" podID="4381712a-9961-4bc9-8b53-be50182422c5" containerID="ee643942d7aa69ccac0fe7d6e754db4262fa6874b72146e9676979dcad5b63ac" exitCode=0
Nov 04 04:01:34.135169 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:34.135044    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs" event={"ID":"4381712a-9961-4bc9-8b53-be50182422c5","Type":"ContainerDied","Data":"ee643942d7aa69ccac0fe7d6e754db4262fa6874b72146e9676979dcad5b63ac"}
Nov 04 04:01:34.238842 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:34.238787    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:34.446710 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:34.446606    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"]
Nov 04 04:01:34.446900 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:34.446873    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy" containerID="cri-o://9b38427be7daa1244daae3f815dfd87a1a62c0eb5f5c9bb4613ae3da13469804" gracePeriod=30
Nov 04 04:01:34.456351 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:34.456324    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:34.720509 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:34.720411    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:35.274878 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.274853    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:35.307554 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.307532    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:35.362243 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.362216    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-openshift-ca\") pod \"59ace1c6-1907-424a-a071-5d7fd00aea0d\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") "
Nov 04 04:01:35.362407 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.362263    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59ace1c6-1907-424a-a071-5d7fd00aea0d-kube-api-access-tdrp4\" (UniqueName: \"kubernetes.io/projected/59ace1c6-1907-424a-a071-5d7fd00aea0d-kube-api-access-tdrp4\") pod \"59ace1c6-1907-424a-a071-5d7fd00aea0d\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") "
Nov 04 04:01:35.362407 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.362299    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-openshift-ca\") pod \"4381712a-9961-4bc9-8b53-be50182422c5\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") "
Nov 04 04:01:35.362539 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.362489    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4381712a-9961-4bc9-8b53-be50182422c5-kube-api-access-ght2k\" (UniqueName: \"kubernetes.io/projected/4381712a-9961-4bc9-8b53-be50182422c5-kube-api-access-ght2k\") pod \"4381712a-9961-4bc9-8b53-be50182422c5\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") "
Nov 04 04:01:35.362598 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.362575    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-openshift-ca" pod "59ace1c6-1907-424a-a071-5d7fd00aea0d" (UID: "59ace1c6-1907-424a-a071-5d7fd00aea0d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:35.362598 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.362582    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-openshift-ca" pod "4381712a-9961-4bc9-8b53-be50182422c5" (UID: "4381712a-9961-4bc9-8b53-be50182422c5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:35.362714 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.362597    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-xks-ca\") pod \"59ace1c6-1907-424a-a071-5d7fd00aea0d\" (UID: \"59ace1c6-1907-424a-a071-5d7fd00aea0d\") "
Nov 04 04:01:35.362752 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.362727    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-xks-ca\") pod \"4381712a-9961-4bc9-8b53-be50182422c5\" (UID: \"4381712a-9961-4bc9-8b53-be50182422c5\") "
Nov 04 04:01:35.363664 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.363161    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:35.363664 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.363181    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:35.364766 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.364737    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4381712a-9961-4bc9-8b53-be50182422c5-kube-api-access-ght2k" pod "4381712a-9961-4bc9-8b53-be50182422c5" (UID: "4381712a-9961-4bc9-8b53-be50182422c5"). InnerVolumeSpecName "kube-api-access-ght2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:35.364766 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.364749    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-xks-ca" pod "59ace1c6-1907-424a-a071-5d7fd00aea0d" (UID: "59ace1c6-1907-424a-a071-5d7fd00aea0d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:35.364969 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.364944    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-xks-ca" pod "4381712a-9961-4bc9-8b53-be50182422c5" (UID: "4381712a-9961-4bc9-8b53-be50182422c5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:35.364969 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.364960    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ace1c6-1907-424a-a071-5d7fd00aea0d-kube-api-access-tdrp4" pod "59ace1c6-1907-424a-a071-5d7fd00aea0d" (UID: "59ace1c6-1907-424a-a071-5d7fd00aea0d"). InnerVolumeSpecName "kube-api-access-tdrp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:35.467417 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.467241    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4381712a-9961-4bc9-8b53-be50182422c5-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:35.467417 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.467285    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdrp4\" (UniqueName: \"kubernetes.io/projected/59ace1c6-1907-424a-a071-5d7fd00aea0d-kube-api-access-tdrp4\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:35.467417 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.467303    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ght2k\" (UniqueName: \"kubernetes.io/projected/4381712a-9961-4bc9-8b53-be50182422c5-kube-api-access-ght2k\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:35.467417 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.467316    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/59ace1c6-1907-424a-a071-5d7fd00aea0d-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:35.493139 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.493102    6751 patch_prober.go:29] interesting pod/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 04:01:35.493299 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.493169    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 04:01:35.579083 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.579045    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"]
Nov 04 04:01:35.579750 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.579696    6751 cpu_manager.go:414] "containerMap: removing container" podUID="59ace1c6-1907-424a-a071-5d7fd00aea0d" containerName="revoke-keys"
Nov 04 04:01:35.579750 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.579716    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59ace1c6-1907-424a-a071-5d7fd00aea0d" containerName="revoke-keys"
Nov 04 04:01:35.579750 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.579731    6751 cpu_manager.go:414] "containerMap: removing container" podUID="df952825-2dfe-4016-bf56-dd65d66aa8f1" containerName="payload-pre-processing"
Nov 04 04:01:35.579750 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.579740    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df952825-2dfe-4016-bf56-dd65d66aa8f1" containerName="payload-pre-processing"
Nov 04 04:01:35.580049 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.579785    6751 cpu_manager.go:414] "containerMap: removing container" podUID="4381712a-9961-4bc9-8b53-be50182422c5" containerName="revoke-keys"
Nov 04 04:01:35.580049 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.579792    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4381712a-9961-4bc9-8b53-be50182422c5" containerName="revoke-keys"
Nov 04 04:01:35.580049 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.579881    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="59ace1c6-1907-424a-a071-5d7fd00aea0d" containerName="revoke-keys"
Nov 04 04:01:35.580049 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.579890    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="df952825-2dfe-4016-bf56-dd65d66aa8f1" containerName="payload-pre-processing"
Nov 04 04:01:35.580049 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.579907    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="4381712a-9961-4bc9-8b53-be50182422c5" containerName="revoke-keys"
Nov 04 04:01:35.585652 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.583980    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.616851 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.616809    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"]
Nov 04 04:01:35.668443 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.668410    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4697d051-eb18-4ed0-92c9-408a109a5bf1-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.668596 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.668450    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plj2r\" (UniqueName: \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-kube-api-access-plj2r\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.668596 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.668471    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.668596 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.668518    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.668725 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.668702    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.668762 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.668736    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.668816 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.668799    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.668859 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.668831    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.668899 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.668873    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.675175 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.675146    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:35.687211 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.687182    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"]
Nov 04 04:01:35.690901 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.690880    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.705548 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.705524    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"]
Nov 04 04:01:35.769506 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.769441    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4697d051-eb18-4ed0-92c9-408a109a5bf1-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.769506 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.769476    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-istio-token\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.769763 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.769503    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x69bf\" (UniqueName: \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-kube-api-access-x69bf\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.769763 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.769582    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plj2r\" (UniqueName: \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-kube-api-access-plj2r\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.769763 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.769654    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.769763 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.769685    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.769763 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.769721    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56656b54-8f49-4a46-92a6-59400b873509-istiod-ca-cert\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.770009 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.769768    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-credential-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.770009 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.769868    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-data\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.770009 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.769904    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56656b54-8f49-4a46-92a6-59400b873509-istio-podinfo\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.770171 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770010    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.770171 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770054    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.770171 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770108    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.770322 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770244    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4697d051-eb18-4ed0-92c9-408a109a5bf1-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.770322 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770260    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.770322 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770292    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-certs\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.770477 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770327    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.770477 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770460    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.770568 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770506    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-envoy\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.770568 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770543    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.770568 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770545    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.770706 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770653    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.770873 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.770848    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.772238 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.772217    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.772663 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.772611    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.782345 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.782322    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plj2r\" (UniqueName: \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-kube-api-access-plj2r\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.782569 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.782553    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.789004 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.788980    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"]
Nov 04 04:01:35.794077 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.794059    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.809970 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.809932    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"]
Nov 04 04:01:35.872139 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872110    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-envoy\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.872298 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872147    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.872298 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872242    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-istio-token\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.872298 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872276    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x69bf\" (UniqueName: \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-kube-api-access-x69bf\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.872473 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872310    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56656b54-8f49-4a46-92a6-59400b873509-istiod-ca-cert\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.872473 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872328    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.872473 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872348    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.872473 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872383    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-credential-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.872473 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872406    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.872684 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872543    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4tms\" (UniqueName: \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-kube-api-access-q4tms\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.872684 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872568    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.872759 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872701    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-data\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.872759 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872735    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56656b54-8f49-4a46-92a6-59400b873509-istio-podinfo\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.872759 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872746    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-credential-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.872852 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872763    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.872852 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872794    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.872911 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872879    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.872944 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872914    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-certs\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.872944 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872924    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-data\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.873006 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.872957    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.873086 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.873067    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56656b54-8f49-4a46-92a6-59400b873509-istiod-ca-cert\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.873183 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.873166    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-certs\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.873217 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.873175    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-socket\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.874378 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.874358    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-envoy\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.875232 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.875215    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56656b54-8f49-4a46-92a6-59400b873509-istio-podinfo\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.884398 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.884378    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-istio-token\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.884714 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.884697    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x69bf\" (UniqueName: \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-kube-api-access-x69bf\") pod \"e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") " pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:35.945061 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.945039    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:35.973772 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.973742    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.973897 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.973827    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.973958 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.973928    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.974013 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.973958    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.974323 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.974275    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-certs\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.974423 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.974346    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-credential-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.974423 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.974388    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.974560 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.974536    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istiod-ca-cert\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.974696 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.974544    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-socket\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.974696 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.974605    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4tms\" (UniqueName: \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-kube-api-access-q4tms\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.974827 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.974724    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.974827 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.974776    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.974827 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.974805    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.975084 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.975061    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-data\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.976486 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.976466    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-podinfo\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.976762 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.976744    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-envoy\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.987214 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.986760    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-token\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:35.987702 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:35.987681    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4tms\" (UniqueName: \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-kube-api-access-q4tms\") pod \"e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") " pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:36.005149 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.004955    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:36.116679 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.116651    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:36.155356 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.155326    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs"
Nov 04 04:01:36.155520 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.155350    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66f93416-k5vfs" event={"ID":"4381712a-9961-4bc9-8b53-be50182422c5","Type":"ContainerDied","Data":"b6db082d70d855ee4a32a0359b1bed5edfc30ef0330b5ab320a3f40c0113623b"}
Nov 04 04:01:36.155520 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.155446    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6db082d70d855ee4a32a0359b1bed5edfc30ef0330b5ab320a3f40c0113623b"
Nov 04 04:01:36.157127 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.157103    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd" event={"ID":"59ace1c6-1907-424a-a071-5d7fd00aea0d","Type":"ContainerDied","Data":"40ffa8e5e8cc577447dbaa13818633189a206260339297f1fd84bbd412e08834"}
Nov 04 04:01:36.157234 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.157147    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40ffa8e5e8cc577447dbaa13818633189a206260339297f1fd84bbd412e08834"
Nov 04 04:01:36.157234 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.157212    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-464e2bd6-fmfxd"
Nov 04 04:01:36.240392 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.240363    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"]
Nov 04 04:01:36.241687 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:36.241654    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4697d051_eb18_4ed0_92c9_408a109a5bf1.slice/crio-2b5ef7614848d42c8566f33a3a25bd8e262e1ddb3a8095ea26c69df238c75a3a WatchSource:0}: Error finding container 2b5ef7614848d42c8566f33a3a25bd8e262e1ddb3a8095ea26c69df238c75a3a: Status 404 returned error can't find the container with id 2b5ef7614848d42c8566f33a3a25bd8e262e1ddb3a8095ea26c69df238c75a3a
Nov 04 04:01:36.244017 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.243984    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:36.244112 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.244059    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:36.244172 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.244128    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:36.302463 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.302437    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"]
Nov 04 04:01:36.303080 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:36.303054    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56656b54_8f49_4a46_92a6_59400b873509.slice/crio-f1bbff2e323241bae01b378a2621ce3cadf2d65ecf63926e69e3d1f4a9d2931e WatchSource:0}: Error finding container f1bbff2e323241bae01b378a2621ce3cadf2d65ecf63926e69e3d1f4a9d2931e: Status 404 returned error can't find the container with id f1bbff2e323241bae01b378a2621ce3cadf2d65ecf63926e69e3d1f4a9d2931e
Nov 04 04:01:36.305351 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.305319    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:36.305447 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.305394    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:36.305447 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.305441    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:36.406039 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:36.406012    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ceb3c9_eee7_4329_ae34_2dc9191b1908.slice/crio-c986971735fa4c989da106ee485489895c44dc0c53a6d41f8958a92228db7662 WatchSource:0}: Error finding container c986971735fa4c989da106ee485489895c44dc0c53a6d41f8958a92228db7662: Status 404 returned error can't find the container with id c986971735fa4c989da106ee485489895c44dc0c53a6d41f8958a92228db7662
Nov 04 04:01:36.407848 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.407818    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"]
Nov 04 04:01:36.409733 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.408941    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:36.409993 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.409965    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:36.410152 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.410132    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:36.455899 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.455874    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"]
Nov 04 04:01:36.640421 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.639182    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:36.867646 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:36.863955    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:37.018704 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.018056    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"]
Nov 04 04:01:37.066648 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.061020    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"]
Nov 04 04:01:37.066648 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.061159    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.080324 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.079970    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2wnx7\""
Nov 04 04:01:37.080324 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.080112    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 04:01:37.080324 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.080191    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 04:01:37.081266 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.080799    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 04:01:37.109261 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.107844    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-687rg\" (UniqueName: \"kubernetes.io/projected/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-kube-api-access-687rg\") pod \"maas-api-revoke-sub-92bab82f-nvbrz\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.109261 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.107904    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-92bab82f-nvbrz\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.109261 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.108004    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-92bab82f-nvbrz\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.110880 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.109693    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:37.238646 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.212344    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-92bab82f-nvbrz\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.238646 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.212477    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-92bab82f-nvbrz\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.238646 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.212687    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-687rg\" (UniqueName: \"kubernetes.io/projected/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-kube-api-access-687rg\") pod \"maas-api-revoke-sub-92bab82f-nvbrz\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.246856 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.243377    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-687rg\" (UniqueName: \"kubernetes.io/projected/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-kube-api-access-687rg\") pod \"maas-api-revoke-sub-92bab82f-nvbrz\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.246856 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.244147    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-92bab82f-nvbrz\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.246856 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.246009    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-92bab82f-nvbrz\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.315823 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.313368    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" event={"ID":"56656b54-8f49-4a46-92a6-59400b873509","Type":"ContainerStarted","Data":"80ac47283b18d293f6184a2d5b21156ffa625c313c421490cfba2dbb208b893c"}
Nov 04 04:01:37.315823 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.313434    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" event={"ID":"56656b54-8f49-4a46-92a6-59400b873509","Type":"ContainerStarted","Data":"f1bbff2e323241bae01b378a2621ce3cadf2d65ecf63926e69e3d1f4a9d2931e"}
Nov 04 04:01:37.337491 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.336919    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" event={"ID":"4697d051-eb18-4ed0-92c9-408a109a5bf1","Type":"ContainerStarted","Data":"d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124"}
Nov 04 04:01:37.337491 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.336985    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" event={"ID":"4697d051-eb18-4ed0-92c9-408a109a5bf1","Type":"ContainerStarted","Data":"2b5ef7614848d42c8566f33a3a25bd8e262e1ddb3a8095ea26c69df238c75a3a"}
Nov 04 04:01:37.339452 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.339416    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" event={"ID":"74ceb3c9-eee7-4329-ae34-2dc9191b1908","Type":"ContainerStarted","Data":"bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769"}
Nov 04 04:01:37.339588 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.339474    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" event={"ID":"74ceb3c9-eee7-4329-ae34-2dc9191b1908","Type":"ContainerStarted","Data":"c986971735fa4c989da106ee485489895c44dc0c53a6d41f8958a92228db7662"}
Nov 04 04:01:37.342393 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.342348    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" podStartSLOduration=2.342335078 podStartE2EDuration="2.342335078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:37.341044023 +0000 UTC m=+691.674981898" watchObservedRunningTime="2026-11-04 04:01:37.342335078 +0000 UTC m=+691.676272955"
Nov 04 04:01:37.382682 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.376114    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" podStartSLOduration=2.376093377 podStartE2EDuration="2.376093377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:37.374116069 +0000 UTC m=+691.708053993" watchObservedRunningTime="2026-11-04 04:01:37.376093377 +0000 UTC m=+691.710031257"
Nov 04 04:01:37.509643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.505827    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:37.703671 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.689486    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:37.923518 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.923129    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:37.948638 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.948591    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:37.948975 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.948949    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 04:01:37.949144 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:37.949123    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 04:01:38.007687 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.007652    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:38.008049 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.008024    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 04:01:38.008225 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.008202    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 04:01:38.149772 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.149746    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:38.150498 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.150470    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 04:01:38.150687 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.150667    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 04:01:38.184719 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.162696    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" podStartSLOduration=3.16267743 podStartE2EDuration="3.16267743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:37.432952836 +0000 UTC m=+691.766890714" watchObservedRunningTime="2026-11-04 04:01:38.16267743 +0000 UTC m=+692.496615315"
Nov 04 04:01:38.186039 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.186001    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"]
Nov 04 04:01:38.186445 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.186414    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy" containerID="cri-o://ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1" gracePeriod=30
Nov 04 04:01:38.341510 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.218269    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:38.814411 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.777437    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"]
Nov 04 04:01:38.849553 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:38.849477    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:39.029069 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.028954    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 04:01:39.029408 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.029377    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 04:01:39.095537 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.095497    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:39.099770 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.099746    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 04:01:39.099963 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.099942    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 04:01:39.117958 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.117930    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 04:01:39.118162 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.118137    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 04:01:39.151644 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.137764    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"]
Nov 04 04:01:39.152510 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.152457    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy" containerID="cri-o://98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e" gracePeriod=30
Nov 04 04:01:39.738954 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.738816    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:39.876046 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.875353    6751 generic.go:356] "Generic (PLEG): container finished" podID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerID="9b38427be7daa1244daae3f815dfd87a1a62c0eb5f5c9bb4613ae3da13469804" exitCode=0
Nov 04 04:01:39.876046 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.875576    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" event={"ID":"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc","Type":"ContainerDied","Data":"9b38427be7daa1244daae3f815dfd87a1a62c0eb5f5c9bb4613ae3da13469804"}
Nov 04 04:01:39.876046 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.875662    6751 scope.go:122] "RemoveContainer" containerID="d8385c0c0f04f62f88fd7a24e1e8c35a3a92e6f177c4bbb88321913a5eaa5ff1"
Nov 04 04:01:39.942697 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.942662    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz" event={"ID":"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f","Type":"ContainerStarted","Data":"1e8ceb848103e0879b86b5634fc8a61839814b596dc4a18b7d47ced3d91b5d8a"}
Nov 04 04:01:39.942858 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.942723    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz" event={"ID":"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f","Type":"ContainerStarted","Data":"65ee578e8421f09cf3937ed3dc4ef97fe228c32cb814f5116e9c4545a5036137"}
Nov 04 04:01:39.945994 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.945963    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 04:01:39.946148 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:39.946012    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 04:01:40.040406 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.017951    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz" podStartSLOduration=4.017928305 podStartE2EDuration="4.017928305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:39.984478607 +0000 UTC m=+694.318416486" watchObservedRunningTime="2026-11-04 04:01:40.017928305 +0000 UTC m=+694.351866184"
Nov 04 04:01:40.118648 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.111873    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 04:01:40.118648 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.111935    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 04:01:40.143643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.136277    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:40.169842 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.153955    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 04:01:40.169842 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.154008    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 04:01:40.344697 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.344308    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 04:01:40.345791 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.345771    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:40.467643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.462005    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-certs\") pod \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") "
Nov 04 04:01:40.467643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.462115    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-socket\") pod \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") "
Nov 04 04:01:40.467643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.462217    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-credential-socket\") pod \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") "
Nov 04 04:01:40.467643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.462271    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-kube-api-access-hzkgt\" (UniqueName: \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-kube-api-access-hzkgt\") pod \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") "
Nov 04 04:01:40.467643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.462322    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-podinfo\") pod \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") "
Nov 04 04:01:40.467643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.462349    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-token\" (UniqueName: \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-token\") pod \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") "
Nov 04 04:01:40.467643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.462416    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-envoy\") pod \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") "
Nov 04 04:01:40.467643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.462456    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-data\") pod \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") "
Nov 04 04:01:40.467643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.462518    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istiod-ca-cert\") pod \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\" (UID: \"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc\") "
Nov 04 04:01:40.491885 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.491841    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-socket" pod "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" (UID: "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:40.552680 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.500819    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-certs" pod "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" (UID: "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:40.552680 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.501069    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-credential-socket" pod "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" (UID: "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:40.552680 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.535492    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istiod-ca-cert" pod "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" (UID: "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:40.553597 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.553566    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-data" pod "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" (UID: "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:40.589644 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.588737    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:40.596687 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.591069    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:40.596687 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.591102    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:40.596687 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.591117    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:40.596687 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.591134    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:40.596687 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.591150    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:40.658534 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.658490    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-envoy" pod "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" (UID: "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:40.660485 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.660450    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-token" pod "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" (UID: "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:40.694815 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.694777    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-kube-api-access-hzkgt" pod "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" (UID: "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc"). InnerVolumeSpecName "kube-api-access-hzkgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:40.701083 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.701052    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzkgt\" (UniqueName: \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-kube-api-access-hzkgt\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:40.701242 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.701227    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:40.701336 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.701324    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:40.705199 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.704735    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-podinfo" pod "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" (UID: "cb601bd1-0bb6-4c68-b5b6-5955e3763ecc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:01:40.810802 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.810768    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:40.939150 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.939064    6751 generic.go:356] "Generic (PLEG): container finished" podID="06128bb3-f48f-4901-8c9e-4d1f0e24cb1f" containerID="1e8ceb848103e0879b86b5634fc8a61839814b596dc4a18b7d47ced3d91b5d8a" exitCode=0
Nov 04 04:01:40.939150 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.939113    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz" event={"ID":"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f","Type":"ContainerDied","Data":"1e8ceb848103e0879b86b5634fc8a61839814b596dc4a18b7d47ced3d91b5d8a"}
Nov 04 04:01:40.940331 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.940312    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh" event={"ID":"cb601bd1-0bb6-4c68-b5b6-5955e3763ecc","Type":"ContainerDied","Data":"6377d906d271fe1c7cc78c54eb2c9a431962506313545ee27020cb9e01d51bbc"}
Nov 04 04:01:40.940535 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.940517    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"
Nov 04 04:01:40.940674 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.940658    6751 scope.go:122] "RemoveContainer" containerID="9b38427be7daa1244daae3f815dfd87a1a62c0eb5f5c9bb4613ae3da13469804"
Nov 04 04:01:40.945363 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.945335    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 04:01:40.945475 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:40.945371    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 04:01:41.028270 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.027979    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 04:01:41.028270 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.028038    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 04:01:41.040129 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.040100    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"]
Nov 04 04:01:41.043096 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.043074    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-998997-openshift-default-768df74449-8f6fh"]
Nov 04 04:01:41.128749 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.128715    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 04:01:41.128909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.128769    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 04:01:41.137253 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.137138    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj" podUID="70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" containerName="main" containerID="cri-o://8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef" gracePeriod=45
Nov 04 04:01:41.446023 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.445924    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:41.719643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.710503    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:41.745963 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.745932    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:41.850642 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.849596    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" path="/var/lib/kubelet/pods/cb601bd1-0bb6-4c68-b5b6-5955e3763ecc/volumes"
Nov 04 04:01:41.939657 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.938864    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:41.945587 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.945426    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 04:01:41.945587 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.945495    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 04:01:41.945587 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.945517    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-dshm\") pod \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") "
Nov 04 04:01:41.945587 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.945543    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-kube-api-access-n6t7b\" (UniqueName: \"kubernetes.io/projected/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-kube-api-access-n6t7b\") pod \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") "
Nov 04 04:01:41.945587 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.945583    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tmp-dir\") pod \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") "
Nov 04 04:01:41.945902 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.945604    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tls-certs\" (UniqueName: \"kubernetes.io/secret/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tls-certs\") pod \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") "
Nov 04 04:01:41.945902 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.945666    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-home\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-home\") pod \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") "
Nov 04 04:01:41.945902 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.945685    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-model-cache\") pod \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\" (UID: \"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0\") "
Nov 04 04:01:41.946016 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.945995    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tmp-dir" pod "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" (UID: "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:41.946262 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.946218    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-model-cache" pod "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" (UID: "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:41.946371 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.946279    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-home" pod "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" (UID: "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:41.948191 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.948170    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tls-certs" pod "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" (UID: "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:41.948424 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.948400    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-kube-api-access-n6t7b" pod "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" (UID: "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0"). InnerVolumeSpecName "kube-api-access-n6t7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:41.948507 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:41.948444    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-dshm" pod "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" (UID: "70f1bcac-af6a-4019-8fbe-f8a9eb587ba0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:42.012694 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.008642    6751 generic.go:356] "Generic (PLEG): container finished" podID="70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" containerID="8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef" exitCode=0
Nov 04 04:01:42.046173 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.046142    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"
Nov 04 04:01:42.048536 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.048508    6751 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-dshm\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:42.048536 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.048536    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6t7b\" (UniqueName: \"kubernetes.io/projected/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-kube-api-access-n6t7b\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:42.048773 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.048567    6751 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tmp-dir\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:42.048773 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.048578    6751 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-tls-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:42.048773 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.048591    6751 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-home\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:42.048773 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.048606    6751 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0-model-cache\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:42.049533 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.048967    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 04:01:42.049533 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.049028    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 04:01:42.085234 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.049832    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj" event={"ID":"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0","Type":"ContainerDied","Data":"8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef"}
Nov 04 04:01:42.085234 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.049900    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj" event={"ID":"70f1bcac-af6a-4019-8fbe-f8a9eb587ba0","Type":"ContainerDied","Data":"78e2313540ba21b43fbcb8eb9095ee6d6be6a5667d9f7a16884a265306306198"}
Nov 04 04:01:42.085234 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.049926    6751 scope.go:122] "RemoveContainer" containerID="8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef"
Nov 04 04:01:42.119658 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.118294    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 04:01:42.119658 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.118361    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 04:01:42.140121 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.134547    6751 patch_prober.go:29] interesting pod/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 04:01:42.140121 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.134658    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 04:01:42.205728 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.201468    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:42.232485 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.211130    6751 scope.go:122] "RemoveContainer" containerID="8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef"
Nov 04 04:01:42.236654 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:42.234402    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef\": container with ID starting with 8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef not found: ID does not exist" containerID="8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef"
Nov 04 04:01:42.236654 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.234467    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef"} err="failed to get container status \"8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef\": rpc error: code = NotFound desc = could not find container \"8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef\": container with ID starting with 8afa144fc6353458d0e6005407106fba265cca42439c9551309edd52372c07ef not found: ID does not exist"
Nov 04 04:01:42.247988 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.247963    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"]
Nov 04 04:01:42.248154 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.248143    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-56a44f/distinct-2-w5-56a44f-kserve-55f845b7f5-wjdmj"]
Nov 04 04:01:42.791671 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.791355    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:42.917129 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.917033    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-openshift-ca\") pod \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") "
Nov 04 04:01:42.919007 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.917779    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-xks-ca\") pod \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") "
Nov 04 04:01:42.919007 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.917889    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-kube-api-access-687rg\" (UniqueName: \"kubernetes.io/projected/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-kube-api-access-687rg\") pod \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\" (UID: \"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f\") "
Nov 04 04:01:42.919349 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.919325    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-openshift-ca" pod "06128bb3-f48f-4901-8c9e-4d1f0e24cb1f" (UID: "06128bb3-f48f-4901-8c9e-4d1f0e24cb1f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:42.926746 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.926714    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-xks-ca" pod "06128bb3-f48f-4901-8c9e-4d1f0e24cb1f" (UID: "06128bb3-f48f-4901-8c9e-4d1f0e24cb1f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:42.926857 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.926763    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-kube-api-access-687rg" pod "06128bb3-f48f-4901-8c9e-4d1f0e24cb1f" (UID: "06128bb3-f48f-4901-8c9e-4d1f0e24cb1f"). InnerVolumeSpecName "kube-api-access-687rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:42.945351 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.945323    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 04:01:42.945458 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:42.945380    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 04:01:43.006462 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.006001    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 04:01:43.006462 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.006062    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 04:01:43.022491 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.022404    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:43.022491 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.022438    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-687rg\" (UniqueName: \"kubernetes.io/projected/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-kube-api-access-687rg\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:43.022491 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.022454    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06128bb3-f48f-4901-8c9e-4d1f0e24cb1f-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:43.104904 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.104871    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz" event={"ID":"06128bb3-f48f-4901-8c9e-4d1f0e24cb1f","Type":"ContainerDied","Data":"65ee578e8421f09cf3937ed3dc4ef97fe228c32cb814f5116e9c4545a5036137"}
Nov 04 04:01:43.105106 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.104919    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65ee578e8421f09cf3937ed3dc4ef97fe228c32cb814f5116e9c4545a5036137"
Nov 04 04:01:43.105106 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.104969    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92bab82f-nvbrz"
Nov 04 04:01:43.117079 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.117049    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 04:01:43.117210 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.117099    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 04:01:43.163673 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.160588    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:43.373420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.373374    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:43.405082 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.404954    6751 patch_prober.go:29] interesting pod/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 04:01:43.405612 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.405037    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 04:01:43.413287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.413261    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"]
Nov 04 04:01:43.413453 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.413433    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy" containerID="cri-o://f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490" gracePeriod=30
Nov 04 04:01:43.592662 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.584508    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:43.776983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.776950    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 04:01:43.785906 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.785878    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" path="/var/lib/kubelet/pods/70f1bcac-af6a-4019-8fbe-f8a9eb587ba0/volumes"
Nov 04 04:01:43.934065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.933984    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-certs\") pod \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") "
Nov 04 04:01:43.934527 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934064    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istiod-ca-cert\") pod \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") "
Nov 04 04:01:43.934527 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934137    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-podinfo\") pod \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") "
Nov 04 04:01:43.934527 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934265    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-token\" (UniqueName: \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-token\") pod \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") "
Nov 04 04:01:43.934527 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934296    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-certs" pod "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" (UID: "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:43.934527 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934318    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-kube-api-access-2nclb\" (UniqueName: \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-kube-api-access-2nclb\") pod \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") "
Nov 04 04:01:43.934527 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934414    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-data\") pod \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") "
Nov 04 04:01:43.934527 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934462    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-credential-socket\") pod \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") "
Nov 04 04:01:43.934527 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934507    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-envoy\") pod \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") "
Nov 04 04:01:43.934970 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934548    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-socket\") pod \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\" (UID: \"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1\") "
Nov 04 04:01:43.934970 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934547    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istiod-ca-cert" pod "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" (UID: "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:43.934970 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.934839    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-data" pod "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" (UID: "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:43.935190 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.935166    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:43.935248 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.935196    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:43.935248 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.935213    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:43.935920 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.935885    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-socket" pod "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" (UID: "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:43.936220 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.936179    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-credential-socket" pod "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" (UID: "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:43.936773 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.936746    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-podinfo" pod "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" (UID: "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:01:43.937955 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.937932    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-kube-api-access-2nclb" pod "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" (UID: "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1"). InnerVolumeSpecName "kube-api-access-2nclb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:43.938047 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.937962    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-token" pod "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" (UID: "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:43.939721 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.939696    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-envoy" pod "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" (UID: "2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:43.946343 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.946318    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 04:01:43.946430 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:43.946373    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 04:01:44.006366 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.006336    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 04:01:44.006476 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.006377    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 04:01:44.037025 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.036908    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:44.037025 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.036942    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:44.037025 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.036958    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nclb\" (UniqueName: \"kubernetes.io/projected/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-kube-api-access-2nclb\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:44.037025 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.036973    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:44.037025 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.036988    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:44.037025 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.037002    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:44.097699 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.097664    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:44.114539 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.114506    6751 generic.go:356] "Generic (PLEG): container finished" podID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerID="ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1" exitCode=0
Nov 04 04:01:44.114710 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.114557    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"
Nov 04 04:01:44.114710 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.114592    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" event={"ID":"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1","Type":"ContainerDied","Data":"ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1"}
Nov 04 04:01:44.114710 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.114678    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h" event={"ID":"2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1","Type":"ContainerDied","Data":"fb51d8d22573054fcd2e5f8981a4fa6f70a45b2ee55241e1ca2b6d6a43f66f8d"}
Nov 04 04:01:44.114905 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.114712    6751 scope.go:122] "RemoveContainer" containerID="ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1"
Nov 04 04:01:44.117066 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.117042    6751 patch_prober.go:29] interesting pod/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 04:01:44.117177 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.117100    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 04:01:44.128928 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.128905    6751 scope.go:122] "RemoveContainer" containerID="ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1"
Nov 04 04:01:44.129244 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:44.129218    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1\": container with ID starting with ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1 not found: ID does not exist" containerID="ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1"
Nov 04 04:01:44.129347 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.129255    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1"} err="failed to get container status \"ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1\": rpc error: code = NotFound desc = could not find container \"ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1\": container with ID starting with ce2453169d39e8e69dc72f2ed04f5257c608142a90eabc8212e95a2c96573cf1 not found: ID does not exist"
Nov 04 04:01:44.150432 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.150400    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"]
Nov 04 04:01:44.154706 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.154685    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-6a9cdb-openshift-default-6779d6d555-j4t9h"]
Nov 04 04:01:44.315824 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.315594    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:44.474870 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.467194    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz" podUID="810c0166-1a84-4d90-9ec1-007dea6a282a" containerName="main" containerID="cri-o://2b51f6e6eb9c5fd7fc45dc421be2d3bcd7310ef9f94c54d19b8678605081d65d" gracePeriod=45
Nov 04 04:01:44.496610 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.495998    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx" podUID="992641c2-8001-44ff-a1b7-20401c15780c" containerName="main" containerID="cri-o://cb6aa677ad0f7280720486dadcfde38e9a5be7c9e6ee60d54cc18d9e32574a32" gracePeriod=45
Nov 04 04:01:44.706528 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.706423    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:44.719182 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.717833    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 04:01:44.905909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.897180    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-envoy\") pod \"561d0a19-dce6-44d7-a998-86df9faaceb5\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") "
Nov 04 04:01:44.905909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.897264    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-token\" (UniqueName: \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-token\") pod \"561d0a19-dce6-44d7-a998-86df9faaceb5\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") "
Nov 04 04:01:44.905909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.897341    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-kube-api-access-vwdrh\" (UniqueName: \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-kube-api-access-vwdrh\") pod \"561d0a19-dce6-44d7-a998-86df9faaceb5\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") "
Nov 04 04:01:44.905909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.897422    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/561d0a19-dce6-44d7-a998-86df9faaceb5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/561d0a19-dce6-44d7-a998-86df9faaceb5-istiod-ca-cert\") pod \"561d0a19-dce6-44d7-a998-86df9faaceb5\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") "
Nov 04 04:01:44.905909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.897535    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-data\") pod \"561d0a19-dce6-44d7-a998-86df9faaceb5\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") "
Nov 04 04:01:44.905909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.897669    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-podinfo\") pod \"561d0a19-dce6-44d7-a998-86df9faaceb5\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") "
Nov 04 04:01:44.905909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.897766    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-credential-socket\") pod \"561d0a19-dce6-44d7-a998-86df9faaceb5\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") "
Nov 04 04:01:44.905909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.897853    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-certs\") pod \"561d0a19-dce6-44d7-a998-86df9faaceb5\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") "
Nov 04 04:01:44.905909 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.897949    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-socket\") pod \"561d0a19-dce6-44d7-a998-86df9faaceb5\" (UID: \"561d0a19-dce6-44d7-a998-86df9faaceb5\") "
Nov 04 04:01:44.960945 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.959682    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-socket" pod "561d0a19-dce6-44d7-a998-86df9faaceb5" (UID: "561d0a19-dce6-44d7-a998-86df9faaceb5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:44.961775 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.961452    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-data" pod "561d0a19-dce6-44d7-a998-86df9faaceb5" (UID: "561d0a19-dce6-44d7-a998-86df9faaceb5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:44.964095 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.964070    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-credential-socket" pod "561d0a19-dce6-44d7-a998-86df9faaceb5" (UID: "561d0a19-dce6-44d7-a998-86df9faaceb5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:44.964419 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.964390    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-certs" pod "561d0a19-dce6-44d7-a998-86df9faaceb5" (UID: "561d0a19-dce6-44d7-a998-86df9faaceb5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:44.966107 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.966084    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:44.967207 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.967186    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/561d0a19-dce6-44d7-a998-86df9faaceb5-istiod-ca-cert" pod "561d0a19-dce6-44d7-a998-86df9faaceb5" (UID: "561d0a19-dce6-44d7-a998-86df9faaceb5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:44.983451 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:44.983397    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-kube-api-access-vwdrh" pod "561d0a19-dce6-44d7-a998-86df9faaceb5" (UID: "561d0a19-dce6-44d7-a998-86df9faaceb5"). InnerVolumeSpecName "kube-api-access-vwdrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:45.039635 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.039426    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.039635 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.039467    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.039635 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.039484    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.039635 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.039509    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.039635 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.039527    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwdrh\" (UniqueName: \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-kube-api-access-vwdrh\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.039635 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.039543    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/561d0a19-dce6-44d7-a998-86df9faaceb5-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.096447 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.095604    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-envoy" pod "561d0a19-dce6-44d7-a998-86df9faaceb5" (UID: "561d0a19-dce6-44d7-a998-86df9faaceb5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:45.120151 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.101737    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-token" pod "561d0a19-dce6-44d7-a998-86df9faaceb5" (UID: "561d0a19-dce6-44d7-a998-86df9faaceb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:45.120151 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.101825    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-podinfo" pod "561d0a19-dce6-44d7-a998-86df9faaceb5" (UID: "561d0a19-dce6-44d7-a998-86df9faaceb5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:01:45.149492 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.149392    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.149492 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.149424    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.149492 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.149447    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/561d0a19-dce6-44d7-a998-86df9faaceb5-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.154943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.151652    6751 generic.go:356] "Generic (PLEG): container finished" podID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerID="98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e" exitCode=0
Nov 04 04:01:45.154943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.151784    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" event={"ID":"561d0a19-dce6-44d7-a998-86df9faaceb5","Type":"ContainerDied","Data":"98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e"}
Nov 04 04:01:45.154943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.151890    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp" event={"ID":"561d0a19-dce6-44d7-a998-86df9faaceb5","Type":"ContainerDied","Data":"ed95fb68eed5c7f270e127691f19ab11b80ffcfea22d2b1dca272b4e5b4139cd"}
Nov 04 04:01:45.154943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.151935    6751 scope.go:122] "RemoveContainer" containerID="98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e"
Nov 04 04:01:45.154943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.152144    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"
Nov 04 04:01:45.180561 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.180123    6751 generic.go:356] "Generic (PLEG): container finished" podID="810c0166-1a84-4d90-9ec1-007dea6a282a" containerID="2b51f6e6eb9c5fd7fc45dc421be2d3bcd7310ef9f94c54d19b8678605081d65d" exitCode=0
Nov 04 04:01:45.180561 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.180218    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz" event={"ID":"810c0166-1a84-4d90-9ec1-007dea6a282a","Type":"ContainerDied","Data":"2b51f6e6eb9c5fd7fc45dc421be2d3bcd7310ef9f94c54d19b8678605081d65d"}
Nov 04 04:01:45.180561 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.180252    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz" event={"ID":"810c0166-1a84-4d90-9ec1-007dea6a282a","Type":"ContainerDied","Data":"82821c8b2f30c9a77b5c7e287092debc9f306e85dddc7cd5cacf7cf3ad39ad19"}
Nov 04 04:01:45.180561 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.180269    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82821c8b2f30c9a77b5c7e287092debc9f306e85dddc7cd5cacf7cf3ad39ad19"
Nov 04 04:01:45.182314 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.182290    6751 scope.go:122] "RemoveContainer" containerID="7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a"
Nov 04 04:01:45.186798 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.186577    6751 generic.go:356] "Generic (PLEG): container finished" podID="992641c2-8001-44ff-a1b7-20401c15780c" containerID="cb6aa677ad0f7280720486dadcfde38e9a5be7c9e6ee60d54cc18d9e32574a32" exitCode=0
Nov 04 04:01:45.186798 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.186752    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx" event={"ID":"992641c2-8001-44ff-a1b7-20401c15780c","Type":"ContainerDied","Data":"cb6aa677ad0f7280720486dadcfde38e9a5be7c9e6ee60d54cc18d9e32574a32"}
Nov 04 04:01:45.190722 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.188140    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:01:45.209529 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.209461    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:45.244362 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.244336    6751 scope.go:122] "RemoveContainer" containerID="98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e"
Nov 04 04:01:45.251311 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:45.245760    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e\": container with ID starting with 98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e not found: ID does not exist" containerID="98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e"
Nov 04 04:01:45.251311 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.245810    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e"} err="failed to get container status \"98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e\": rpc error: code = NotFound desc = could not find container \"98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e\": container with ID starting with 98ce8fc3ca1addfaa7369733b7d984580a2ee7aebef65e56ae8a788857fc827e not found: ID does not exist"
Nov 04 04:01:45.251311 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.245839    6751 scope.go:122] "RemoveContainer" containerID="7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a"
Nov 04 04:01:45.254225 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.254201    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"]
Nov 04 04:01:45.255457 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:45.255431    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a\": container with ID starting with 7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a not found: ID does not exist" containerID="7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a"
Nov 04 04:01:45.255690 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.255663    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a"} err="failed to get container status \"7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a\": rpc error: code = NotFound desc = could not find container \"7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a\": container with ID starting with 7cbf4208bfb960d9565002aea6817d6c080fea54bfc43e0773b77168dd515e3a not found: ID does not exist"
Nov 04 04:01:45.264544 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.264523    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-36730f-openshift-default-84ffb6c657-ktksp"]
Nov 04 04:01:45.268670 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.268650    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 04:01:45.294274 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.294198    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:45.295235 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.295207    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:45.296254 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.296232    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:01:45.362643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.355914    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:45.362643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.357280    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:45.362643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.360544    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:01:45.362643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.362536    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/810c0166-1a84-4d90-9ec1-007dea6a282a-kube-api-access-s49d9\" (UniqueName: \"kubernetes.io/projected/810c0166-1a84-4d90-9ec1-007dea6a282a-kube-api-access-s49d9\") pod \"810c0166-1a84-4d90-9ec1-007dea6a282a\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") "
Nov 04 04:01:45.362643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.362595    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-dshm\") pod \"810c0166-1a84-4d90-9ec1-007dea6a282a\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") "
Nov 04 04:01:45.362643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.362646    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-home\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-home\") pod \"810c0166-1a84-4d90-9ec1-007dea6a282a\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") "
Nov 04 04:01:45.368047 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.362674    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-model-cache\") pod \"810c0166-1a84-4d90-9ec1-007dea6a282a\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") "
Nov 04 04:01:45.368047 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.362765    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-tmp-dir\") pod \"810c0166-1a84-4d90-9ec1-007dea6a282a\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") "
Nov 04 04:01:45.368047 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.362799    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/810c0166-1a84-4d90-9ec1-007dea6a282a-tls-certs\" (UniqueName: \"kubernetes.io/secret/810c0166-1a84-4d90-9ec1-007dea6a282a-tls-certs\") pod \"810c0166-1a84-4d90-9ec1-007dea6a282a\" (UID: \"810c0166-1a84-4d90-9ec1-007dea6a282a\") "
Nov 04 04:01:45.368047 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.364731    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-home" pod "810c0166-1a84-4d90-9ec1-007dea6a282a" (UID: "810c0166-1a84-4d90-9ec1-007dea6a282a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:45.368047 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.365396    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-model-cache" pod "810c0166-1a84-4d90-9ec1-007dea6a282a" (UID: "810c0166-1a84-4d90-9ec1-007dea6a282a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:45.368047 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.366030    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-tmp-dir" pod "810c0166-1a84-4d90-9ec1-007dea6a282a" (UID: "810c0166-1a84-4d90-9ec1-007dea6a282a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:45.370792 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.370765    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/810c0166-1a84-4d90-9ec1-007dea6a282a-tls-certs" pod "810c0166-1a84-4d90-9ec1-007dea6a282a" (UID: "810c0166-1a84-4d90-9ec1-007dea6a282a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:45.371062 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.371042    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/810c0166-1a84-4d90-9ec1-007dea6a282a-kube-api-access-s49d9" pod "810c0166-1a84-4d90-9ec1-007dea6a282a" (UID: "810c0166-1a84-4d90-9ec1-007dea6a282a"). InnerVolumeSpecName "kube-api-access-s49d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:45.372536 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.372515    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-dshm" pod "810c0166-1a84-4d90-9ec1-007dea6a282a" (UID: "810c0166-1a84-4d90-9ec1-007dea6a282a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:45.376367 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.376343    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:45.378764 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.378741    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:45.379728 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.379703    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:01:45.463555 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.463501    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-model-cache\") pod \"992641c2-8001-44ff-a1b7-20401c15780c\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") "
Nov 04 04:01:45.463799 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.463566    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-dshm\") pod \"992641c2-8001-44ff-a1b7-20401c15780c\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") "
Nov 04 04:01:45.463799 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.463631    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-home\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-home\") pod \"992641c2-8001-44ff-a1b7-20401c15780c\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") "
Nov 04 04:01:45.463799 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.463667    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/992641c2-8001-44ff-a1b7-20401c15780c-kube-api-access-gjvxk\" (UniqueName: \"kubernetes.io/projected/992641c2-8001-44ff-a1b7-20401c15780c-kube-api-access-gjvxk\") pod \"992641c2-8001-44ff-a1b7-20401c15780c\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") "
Nov 04 04:01:45.463799 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.463739    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/992641c2-8001-44ff-a1b7-20401c15780c-tls-certs\" (UniqueName: \"kubernetes.io/secret/992641c2-8001-44ff-a1b7-20401c15780c-tls-certs\") pod \"992641c2-8001-44ff-a1b7-20401c15780c\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") "
Nov 04 04:01:45.464039 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.463817    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-tmp-dir\") pod \"992641c2-8001-44ff-a1b7-20401c15780c\" (UID: \"992641c2-8001-44ff-a1b7-20401c15780c\") "
Nov 04 04:01:45.464530 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.464401    6751 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-home\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.464530 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.464432    6751 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-model-cache\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.464530 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.464448    6751 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-tmp-dir\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.464530 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.464464    6751 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/810c0166-1a84-4d90-9ec1-007dea6a282a-tls-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.464530 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.464479    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s49d9\" (UniqueName: \"kubernetes.io/projected/810c0166-1a84-4d90-9ec1-007dea6a282a-kube-api-access-s49d9\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.464530 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.464494    6751 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/810c0166-1a84-4d90-9ec1-007dea6a282a-dshm\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.464530 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.464409    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-model-cache" pod "992641c2-8001-44ff-a1b7-20401c15780c" (UID: "992641c2-8001-44ff-a1b7-20401c15780c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:45.465567 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.465545    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-tmp-dir" pod "992641c2-8001-44ff-a1b7-20401c15780c" (UID: "992641c2-8001-44ff-a1b7-20401c15780c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:45.466545 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.466502    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-home" pod "992641c2-8001-44ff-a1b7-20401c15780c" (UID: "992641c2-8001-44ff-a1b7-20401c15780c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:45.466972 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.466945    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-dshm" pod "992641c2-8001-44ff-a1b7-20401c15780c" (UID: "992641c2-8001-44ff-a1b7-20401c15780c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:45.478028 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.478000    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/992641c2-8001-44ff-a1b7-20401c15780c-kube-api-access-gjvxk" pod "992641c2-8001-44ff-a1b7-20401c15780c" (UID: "992641c2-8001-44ff-a1b7-20401c15780c"). InnerVolumeSpecName "kube-api-access-gjvxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:45.480078 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.480055    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/992641c2-8001-44ff-a1b7-20401c15780c-tls-certs" pod "992641c2-8001-44ff-a1b7-20401c15780c" (UID: "992641c2-8001-44ff-a1b7-20401c15780c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:45.566512 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.566418    6751 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-tmp-dir\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.566512 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.566444    6751 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-model-cache\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.566512 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.566453    6751 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-dshm\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.566512 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.566461    6751 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/992641c2-8001-44ff-a1b7-20401c15780c-home\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.566512 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.566473    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjvxk\" (UniqueName: \"kubernetes.io/projected/992641c2-8001-44ff-a1b7-20401c15780c-kube-api-access-gjvxk\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.566512 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.566486    6751 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/992641c2-8001-44ff-a1b7-20401c15780c-tls-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:45.785804 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.785766    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" path="/var/lib/kubelet/pods/2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1/volumes"
Nov 04 04:01:45.786423 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.786403    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" path="/var/lib/kubelet/pods/561d0a19-dce6-44d7-a998-86df9faaceb5/volumes"
Nov 04 04:01:45.812007 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.811972    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz" podUID="810c0166-1a84-4d90-9ec1-007dea6a282a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 04:01:45.974239 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:45.974183    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx" podUID="992641c2-8001-44ff-a1b7-20401c15780c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 04 04:01:46.129544 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:46.129506    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:46.198010 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:46.197980    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"
Nov 04 04:01:46.198509 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:46.198156    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx" event={"ID":"992641c2-8001-44ff-a1b7-20401c15780c","Type":"ContainerDied","Data":"2fe751cb4e122de756c5d7174e6f4750846754c50dec7dd4f91dc26a0c9ba0f2"}
Nov 04 04:01:46.198729 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:46.198710    6751 scope.go:122] "RemoveContainer" containerID="cb6aa677ad0f7280720486dadcfde38e9a5be7c9e6ee60d54cc18d9e32574a32"
Nov 04 04:01:46.206615 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:46.206586    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"
Nov 04 04:01:46.232110 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:46.232057    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"]
Nov 04 04:01:46.236301 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:46.236281    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/distinct-w4-64c20d-kserve-55cf8dd5f8-6ktzz"]
Nov 04 04:01:46.247690 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:46.247670    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"]
Nov 04 04:01:46.252854 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:46.252829    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-64c20d/simulator-w4-64c20d-kserve-59c486cfbb-d5pfx"]
Nov 04 04:01:47.011667 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:47.011640    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:47.315122 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:47.315049    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:47.622048 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:47.622019    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:47.785439 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:47.785411    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="810c0166-1a84-4d90-9ec1-007dea6a282a" path="/var/lib/kubelet/pods/810c0166-1a84-4d90-9ec1-007dea6a282a/volumes"
Nov 04 04:01:47.785911 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:47.785817    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="992641c2-8001-44ff-a1b7-20401c15780c" path="/var/lib/kubelet/pods/992641c2-8001-44ff-a1b7-20401c15780c/volumes"
Nov 04 04:01:47.840812 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:47.834488    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:48.047821 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:48.047747    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:48.604263 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:48.604231    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:49.075351 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.075321    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 04:01:49.111456 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.111415    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:49.147749 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.147716    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-certs\") pod \"aff9d333-98c8-4671-a057-b07ac7497812\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") "
Nov 04 04:01:49.147938 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.147767    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aff9d333-98c8-4671-a057-b07ac7497812-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff9d333-98c8-4671-a057-b07ac7497812-istio-podinfo\") pod \"aff9d333-98c8-4671-a057-b07ac7497812\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") "
Nov 04 04:01:49.147938 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.147805    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-kube-api-access-5xnmp\" (UniqueName: \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-kube-api-access-5xnmp\") pod \"aff9d333-98c8-4671-a057-b07ac7497812\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") "
Nov 04 04:01:49.147938 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.147837    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aff9d333-98c8-4671-a057-b07ac7497812-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff9d333-98c8-4671-a057-b07ac7497812-istiod-ca-cert\") pod \"aff9d333-98c8-4671-a057-b07ac7497812\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") "
Nov 04 04:01:49.147938 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.147892    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-istio-token\" (UniqueName: \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-istio-token\") pod \"aff9d333-98c8-4671-a057-b07ac7497812\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") "
Nov 04 04:01:49.148160 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.147963    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-credential-socket\") pod \"aff9d333-98c8-4671-a057-b07ac7497812\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") "
Nov 04 04:01:49.148160 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.148008    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-envoy\") pod \"aff9d333-98c8-4671-a057-b07ac7497812\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") "
Nov 04 04:01:49.148160 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.148015    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-certs" pod "aff9d333-98c8-4671-a057-b07ac7497812" (UID: "aff9d333-98c8-4671-a057-b07ac7497812"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:49.148160 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.148095    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-socket\") pod \"aff9d333-98c8-4671-a057-b07ac7497812\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") "
Nov 04 04:01:49.148160 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.148135    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-data\") pod \"aff9d333-98c8-4671-a057-b07ac7497812\" (UID: \"aff9d333-98c8-4671-a057-b07ac7497812\") "
Nov 04 04:01:49.148868 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.148538    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-credential-socket" pod "aff9d333-98c8-4671-a057-b07ac7497812" (UID: "aff9d333-98c8-4671-a057-b07ac7497812"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:49.148868 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.148544    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:49.148868 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.148644    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-socket" pod "aff9d333-98c8-4671-a057-b07ac7497812" (UID: "aff9d333-98c8-4671-a057-b07ac7497812"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:49.148868 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.148746    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-data" pod "aff9d333-98c8-4671-a057-b07ac7497812" (UID: "aff9d333-98c8-4671-a057-b07ac7497812"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:49.148868 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.148833    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff9d333-98c8-4671-a057-b07ac7497812-istiod-ca-cert" pod "aff9d333-98c8-4671-a057-b07ac7497812" (UID: "aff9d333-98c8-4671-a057-b07ac7497812"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:49.150664 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.150631    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-istio-token" pod "aff9d333-98c8-4671-a057-b07ac7497812" (UID: "aff9d333-98c8-4671-a057-b07ac7497812"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:49.150783 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.150677    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aff9d333-98c8-4671-a057-b07ac7497812-istio-podinfo" pod "aff9d333-98c8-4671-a057-b07ac7497812" (UID: "aff9d333-98c8-4671-a057-b07ac7497812"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:01:49.150783 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.150719    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-envoy" pod "aff9d333-98c8-4671-a057-b07ac7497812" (UID: "aff9d333-98c8-4671-a057-b07ac7497812"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:49.150783 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.150774    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-kube-api-access-5xnmp" pod "aff9d333-98c8-4671-a057-b07ac7497812" (UID: "aff9d333-98c8-4671-a057-b07ac7497812"). InnerVolumeSpecName "kube-api-access-5xnmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:49.225252 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.225192    6751 generic.go:356] "Generic (PLEG): container finished" podID="aff9d333-98c8-4671-a057-b07ac7497812" containerID="f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490" exitCode=0
Nov 04 04:01:49.225252 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.225239    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"
Nov 04 04:01:49.225407 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.225270    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" event={"ID":"aff9d333-98c8-4671-a057-b07ac7497812","Type":"ContainerDied","Data":"f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490"}
Nov 04 04:01:49.225407 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.225322    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv" event={"ID":"aff9d333-98c8-4671-a057-b07ac7497812","Type":"ContainerDied","Data":"6703dbc7e3223541ef5c1f8bbd52435d377b4d89d3e81863ca8916931e0cf553"}
Nov 04 04:01:49.225407 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.225345    6751 scope.go:122] "RemoveContainer" containerID="f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490"
Nov 04 04:01:49.235670 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.235654    6751 scope.go:122] "RemoveContainer" containerID="39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9"
Nov 04 04:01:49.258718 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.251514    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:49.258718 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.251553    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:49.258718 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.251574    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:49.258718 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.251587    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:49.258718 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.251602    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff9d333-98c8-4671-a057-b07ac7497812-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:49.258718 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.251632    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff9d333-98c8-4671-a057-b07ac7497812-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:49.258718 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.251647    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xnmp\" (UniqueName: \"kubernetes.io/projected/aff9d333-98c8-4671-a057-b07ac7497812-kube-api-access-5xnmp\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:49.258718 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.251659    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff9d333-98c8-4671-a057-b07ac7497812-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:49.278817 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.278777    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"]
Nov 04 04:01:49.282283 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.282254    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-78ca37-openshift-default-6d9b468c59-sf4jv"]
Nov 04 04:01:49.298173 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.298146    6751 scope.go:122] "RemoveContainer" containerID="f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490"
Nov 04 04:01:49.298541 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:49.298521    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490\": container with ID starting with f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490 not found: ID does not exist" containerID="f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490"
Nov 04 04:01:49.298602 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.298552    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490"} err="failed to get container status \"f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490\": rpc error: code = NotFound desc = could not find container \"f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490\": container with ID starting with f02f691dd104bd2cc421283e458d49778e123657613c3ff1efa185e3bed71490 not found: ID does not exist"
Nov 04 04:01:49.298602 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.298572    6751 scope.go:122] "RemoveContainer" containerID="39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9"
Nov 04 04:01:49.298896 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:49.298879    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9\": container with ID starting with 39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9 not found: ID does not exist" containerID="39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9"
Nov 04 04:01:49.298949 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.298901    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9"} err="failed to get container status \"39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9\": rpc error: code = NotFound desc = could not find container \"39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9\": container with ID starting with 39b5aac35bd671bd4b46a5f96d831bc1a101e27946786637672d7aa85e1493d9 not found: ID does not exist"
Nov 04 04:01:49.326100 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.326078    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:49.600889 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.600864    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"]
Nov 04 04:01:49.601324 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601309    6751 cpu_manager.go:414] "containerMap: removing container" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy"
Nov 04 04:01:49.601376 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601325    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy"
Nov 04 04:01:49.601376 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601338    6751 cpu_manager.go:414] "containerMap: removing container" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy"
Nov 04 04:01:49.601376 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601343    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy"
Nov 04 04:01:49.601376 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601352    6751 cpu_manager.go:414] "containerMap: removing container" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy"
Nov 04 04:01:49.601376 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601357    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy"
Nov 04 04:01:49.601376 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601364    6751 cpu_manager.go:414] "containerMap: removing container" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy"
Nov 04 04:01:49.601376 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601369    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601379    6751 cpu_manager.go:414] "containerMap: removing container" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601384    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601394    6751 cpu_manager.go:414] "containerMap: removing container" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601399    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601412    6751 cpu_manager.go:414] "containerMap: removing container" podUID="70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" containerName="main"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601417    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" containerName="main"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601423    6751 cpu_manager.go:414] "containerMap: removing container" podUID="810c0166-1a84-4d90-9ec1-007dea6a282a" containerName="main"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601428    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="810c0166-1a84-4d90-9ec1-007dea6a282a" containerName="main"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601434    6751 cpu_manager.go:414] "containerMap: removing container" podUID="06128bb3-f48f-4901-8c9e-4d1f0e24cb1f" containerName="revoke-keys"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601438    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06128bb3-f48f-4901-8c9e-4d1f0e24cb1f" containerName="revoke-keys"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601446    6751 cpu_manager.go:414] "containerMap: removing container" podUID="992641c2-8001-44ff-a1b7-20401c15780c" containerName="main"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601449    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="992641c2-8001-44ff-a1b7-20401c15780c" containerName="main"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601505    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="70f1bcac-af6a-4019-8fbe-f8a9eb587ba0" containerName="main"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601511    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="810c0166-1a84-4d90-9ec1-007dea6a282a" containerName="main"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601517    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="06128bb3-f48f-4901-8c9e-4d1f0e24cb1f" containerName="revoke-keys"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601523    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="992641c2-8001-44ff-a1b7-20401c15780c" containerName="main"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601528    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601536    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e4dee6c-a29f-4b6b-a29b-19aa0799c8c1" containerName="istio-proxy"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601540    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601545    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601551    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb601bd1-0bb6-4c68-b5b6-5955e3763ecc" containerName="istio-proxy"
Nov 04 04:01:49.601592 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.601557    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="561d0a19-dce6-44d7-a998-86df9faaceb5" containerName="istio-proxy"
Nov 04 04:01:49.606350 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.606332    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:49.611167 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.611109    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 04:01:49.611820 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.611802    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 04:01:49.612016 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.611976    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 04:01:49.612300 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.612278    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2wnx7\""
Nov 04 04:01:49.615579 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.615557    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"]
Nov 04 04:01:49.655734 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.655706    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxgp5\" (UniqueName: \"kubernetes.io/projected/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-kube-api-access-pxgp5\") pod \"maas-api-revoke-sub-ec7bdfdd-4drjl\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:49.655843 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.655750    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec7bdfdd-4drjl\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:49.655889 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.655874    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec7bdfdd-4drjl\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:49.757022 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.756981    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec7bdfdd-4drjl\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:49.757134 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.757057    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec7bdfdd-4drjl\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:49.757134 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.757114    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxgp5\" (UniqueName: \"kubernetes.io/projected/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-kube-api-access-pxgp5\") pod \"maas-api-revoke-sub-ec7bdfdd-4drjl\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:49.757766 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.757744    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec7bdfdd-4drjl\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:49.759291 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.759270    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec7bdfdd-4drjl\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:49.769221 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.769200    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxgp5\" (UniqueName: \"kubernetes.io/projected/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-kube-api-access-pxgp5\") pod \"maas-api-revoke-sub-ec7bdfdd-4drjl\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:49.784329 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.784307    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aff9d333-98c8-4671-a057-b07ac7497812" path="/var/lib/kubelet/pods/aff9d333-98c8-4671-a057-b07ac7497812/volumes"
Nov 04 04:01:49.922149 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:49.922076    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:50.093385 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.093354    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:50.198685 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.198656    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"]
Nov 04 04:01:50.199998 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:50.199974    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef77f1d6_3638_4e63_ad6c_b0e8ea320df2.slice/crio-5c55d0103f8f07a38277d2bd6ff0ae28fae2bd8ec364d6fc506234d59cc88cfc WatchSource:0}: Error finding container 5c55d0103f8f07a38277d2bd6ff0ae28fae2bd8ec364d6fc506234d59cc88cfc: Status 404 returned error can't find the container with id 5c55d0103f8f07a38277d2bd6ff0ae28fae2bd8ec364d6fc506234d59cc88cfc
Nov 04 04:01:50.233712 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.233689    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl" event={"ID":"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2","Type":"ContainerStarted","Data":"5c55d0103f8f07a38277d2bd6ff0ae28fae2bd8ec364d6fc506234d59cc88cfc"}
Nov 04 04:01:50.612434 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.612403    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"]
Nov 04 04:01:50.612845 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.612832    6751 cpu_manager.go:414] "containerMap: removing container" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy"
Nov 04 04:01:50.612899 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.612846    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy"
Nov 04 04:01:50.612949 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.612941    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="aff9d333-98c8-4671-a057-b07ac7497812" containerName="istio-proxy"
Nov 04 04:01:50.616417 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.616396    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.631262 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.631224    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"]
Nov 04 04:01:50.668079 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.668044    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.668239 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.668105    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9clr\" (UniqueName: \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-kube-api-access-f9clr\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.668315 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.668262    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.668315 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.668302    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.668433 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.668417    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.668489 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.668448    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.668489 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.668480    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52a8de46-9185-43a4-80c5-cfcedf3dd676-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.668656 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.668638    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.668714 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.668698    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.701494 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.701464    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:50.769589 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.769528    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.769589 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.769582    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.769774 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.769609    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9clr\" (UniqueName: \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-kube-api-access-f9clr\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.769774 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.769695    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.769774 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.769716    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.769931 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.769859    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.769931 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.769900    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.770045 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.769939    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52a8de46-9185-43a4-80c5-cfcedf3dd676-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.770045 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.769960    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-data\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.770045 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.769973    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.770045 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.770014    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-certs\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.770229 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.770080    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-credential-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.770229 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.770193    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-socket\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.770631 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.770601    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52a8de46-9185-43a4-80c5-cfcedf3dd676-istiod-ca-cert\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.771973 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.771950    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-envoy\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.772183 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.772164    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-podinfo\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.789561 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.789539    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-token\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.814780 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.802773    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9clr\" (UniqueName: \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-kube-api-access-f9clr\") pod \"e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") " pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.930203 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.930162    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:50.966331 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:50.966297    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:51.086260 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:01:51.086214    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 04:01:51.206112 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:51.206072    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:51.237823 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:51.237790    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"]
Nov 04 04:01:51.240206 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:01:51.239788    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a8de46_9185_43a4_80c5_cfcedf3dd676.slice/crio-07b955da16452896206b54faed0340f69f4f2411d88563f26c44897b32c4f084 WatchSource:0}: Error finding container 07b955da16452896206b54faed0340f69f4f2411d88563f26c44897b32c4f084: Status 404 returned error can't find the container with id 07b955da16452896206b54faed0340f69f4f2411d88563f26c44897b32c4f084
Nov 04 04:01:51.242411 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:51.242378    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:51.242533 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:51.242458    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:51.242533 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:51.242501    6751 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 04:01:51.244346 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:51.244316    6751 generic.go:356] "Generic (PLEG): container finished" podID="ef77f1d6-3638-4e63-ad6c-b0e8ea320df2" containerID="ade785680192e7a8e93ac1dcb3cda67d76c243afc11e87907efce49e9e4dc625" exitCode=0
Nov 04 04:01:51.246761 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:51.244350    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl" event={"ID":"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2","Type":"ContainerDied","Data":"ade785680192e7a8e93ac1dcb3cda67d76c243afc11e87907efce49e9e4dc625"}
Nov 04 04:01:51.509645 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:51.508534    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75" podUID="e1069831-769d-4c65-b634-f6c12d2e00b9" containerName="main" containerID="cri-o://a5cc82d72910ca5ab7a01a6220f76a11bce140d6899a6604dbede9e4eb98fdd7" gracePeriod=45
Nov 04 04:01:52.502672 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.464042    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:52.590369 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.590325    6751 generic.go:356] "Generic (PLEG): container finished" podID="e1069831-769d-4c65-b634-f6c12d2e00b9" containerID="a5cc82d72910ca5ab7a01a6220f76a11bce140d6899a6604dbede9e4eb98fdd7" exitCode=0
Nov 04 04:01:52.609122 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.609085    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75" event={"ID":"e1069831-769d-4c65-b634-f6c12d2e00b9","Type":"ContainerDied","Data":"a5cc82d72910ca5ab7a01a6220f76a11bce140d6899a6604dbede9e4eb98fdd7"}
Nov 04 04:01:52.640504 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.640466    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75" event={"ID":"e1069831-769d-4c65-b634-f6c12d2e00b9","Type":"ContainerDied","Data":"6e8a94dd0c107d54a9dfe065f8160b55d9475a3591a6e9d65d1ddc0e9cccac61"}
Nov 04 04:01:52.640773 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.640757    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e8a94dd0c107d54a9dfe065f8160b55d9475a3591a6e9d65d1ddc0e9cccac61"
Nov 04 04:01:52.640887 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.627914    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:52.713641 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.712606    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" event={"ID":"52a8de46-9185-43a4-80c5-cfcedf3dd676","Type":"ContainerStarted","Data":"6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43"}
Nov 04 04:01:52.713641 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.712690    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" event={"ID":"52a8de46-9185-43a4-80c5-cfcedf3dd676","Type":"ContainerStarted","Data":"07b955da16452896206b54faed0340f69f4f2411d88563f26c44897b32c4f084"}
Nov 04 04:01:52.739663 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.714475    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-tmp-dir\") pod \"e1069831-769d-4c65-b634-f6c12d2e00b9\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") "
Nov 04 04:01:52.739663 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.714552    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1069831-769d-4c65-b634-f6c12d2e00b9-kube-api-access-qmcb6\" (UniqueName: \"kubernetes.io/projected/e1069831-769d-4c65-b634-f6c12d2e00b9-kube-api-access-qmcb6\") pod \"e1069831-769d-4c65-b634-f6c12d2e00b9\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") "
Nov 04 04:01:52.739663 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.714580    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-home\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-home\") pod \"e1069831-769d-4c65-b634-f6c12d2e00b9\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") "
Nov 04 04:01:52.739663 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.714709    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-model-cache\") pod \"e1069831-769d-4c65-b634-f6c12d2e00b9\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") "
Nov 04 04:01:52.739663 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.714745    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1069831-769d-4c65-b634-f6c12d2e00b9-tls-certs\" (UniqueName: \"kubernetes.io/secret/e1069831-769d-4c65-b634-f6c12d2e00b9-tls-certs\") pod \"e1069831-769d-4c65-b634-f6c12d2e00b9\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") "
Nov 04 04:01:52.739663 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.714787    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-dshm\") pod \"e1069831-769d-4c65-b634-f6c12d2e00b9\" (UID: \"e1069831-769d-4c65-b634-f6c12d2e00b9\") "
Nov 04 04:01:52.739663 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.734955    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-model-cache" pod "e1069831-769d-4c65-b634-f6c12d2e00b9" (UID: "e1069831-769d-4c65-b634-f6c12d2e00b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:52.739663 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.735153    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-home" pod "e1069831-769d-4c65-b634-f6c12d2e00b9" (UID: "e1069831-769d-4c65-b634-f6c12d2e00b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:52.750834 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.750780    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-tmp-dir" pod "e1069831-769d-4c65-b634-f6c12d2e00b9" (UID: "e1069831-769d-4c65-b634-f6c12d2e00b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:52.868651 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.867425    6751 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-tmp-dir\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:52.868651 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.867461    6751 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-home\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:52.868651 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.867477    6751 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-model-cache\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:52.938643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.930156    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:52.938643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.935695    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:01:52.985797 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.985756    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 04:01:52.986085 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.986052    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 04:01:52.990643 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:52.988287    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-dshm" pod "e1069831-769d-4c65-b634-f6c12d2e00b9" (UID: "e1069831-769d-4c65-b634-f6c12d2e00b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:01:53.020855 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.002997    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1069831-769d-4c65-b634-f6c12d2e00b9-kube-api-access-qmcb6" pod "e1069831-769d-4c65-b634-f6c12d2e00b9" (UID: "e1069831-769d-4c65-b634-f6c12d2e00b9"). InnerVolumeSpecName "kube-api-access-qmcb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:53.020855 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.004508    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1069831-769d-4c65-b634-f6c12d2e00b9-tls-certs" pod "e1069831-769d-4c65-b634-f6c12d2e00b9" (UID: "e1069831-769d-4c65-b634-f6c12d2e00b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:53.060704 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.059723    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75" podUID="e1069831-769d-4c65-b634-f6c12d2e00b9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 04:01:53.092655 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.088399    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmcb6\" (UniqueName: \"kubernetes.io/projected/e1069831-769d-4c65-b634-f6c12d2e00b9-kube-api-access-qmcb6\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:53.092655 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.088443    6751 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1069831-769d-4c65-b634-f6c12d2e00b9-tls-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:53.092655 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.088458    6751 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1069831-769d-4c65-b634-f6c12d2e00b9-dshm\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:53.773244 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.760908    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"
Nov 04 04:01:53.910576 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.910543    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:53.925514 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.925464    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" podStartSLOduration=3.925449656 podStartE2EDuration="3.925449656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 04:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:01:52.863863592 +0000 UTC m=+707.197801471" watchObservedRunningTime="2026-11-04 04:01:53.925449656 +0000 UTC m=+708.259387534"
Nov 04 04:01:53.928076 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.928050    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"]
Nov 04 04:01:53.931230 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.930560    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 04:01:53.931230 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.930657    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 04:01:53.934058 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:53.934034    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4afe0e/distinct-2-w0-4afe0e-kserve-7b7bfb5bd7-zwj75"]
Nov 04 04:01:54.075210 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.075098    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-kube-api-access-pxgp5\" (UniqueName: \"kubernetes.io/projected/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-kube-api-access-pxgp5\") pod \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") "
Nov 04 04:01:54.075398 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.075213    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-openshift-ca\") pod \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") "
Nov 04 04:01:54.075398 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.075287    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-xks-ca\") pod \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\" (UID: \"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2\") "
Nov 04 04:01:54.092646 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.088846    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-openshift-ca" pod "ef77f1d6-3638-4e63-ad6c-b0e8ea320df2" (UID: "ef77f1d6-3638-4e63-ad6c-b0e8ea320df2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:01:54.095201 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.095171    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-xks-ca" pod "ef77f1d6-3638-4e63-ad6c-b0e8ea320df2" (UID: "ef77f1d6-3638-4e63-ad6c-b0e8ea320df2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 04:01:54.095427 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.095412    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-kube-api-access-pxgp5" pod "ef77f1d6-3638-4e63-ad6c-b0e8ea320df2" (UID: "ef77f1d6-3638-4e63-ad6c-b0e8ea320df2"). InnerVolumeSpecName "kube-api-access-pxgp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:01:54.181716 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.180396    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxgp5\" (UniqueName: \"kubernetes.io/projected/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-kube-api-access-pxgp5\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:54.181716 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.180443    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-openshift-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:54.181716 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.180462    6751 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef77f1d6-3638-4e63-ad6c-b0e8ea320df2-maas-api-xks-ca\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:01:54.789106 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.789079    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl"
Nov 04 04:01:54.789596 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.789126    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec7bdfdd-4drjl" event={"ID":"ef77f1d6-3638-4e63-ad6c-b0e8ea320df2","Type":"ContainerDied","Data":"5c55d0103f8f07a38277d2bd6ff0ae28fae2bd8ec364d6fc506234d59cc88cfc"}
Nov 04 04:01:54.789596 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.789178    6751 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c55d0103f8f07a38277d2bd6ff0ae28fae2bd8ec364d6fc506234d59cc88cfc"
Nov 04 04:01:54.930746 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.930707    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 04:01:54.930925 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:54.930767    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 04:01:55.785786 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:55.785755    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1069831-769d-4c65-b634-f6c12d2e00b9" path="/var/lib/kubelet/pods/e1069831-769d-4c65-b634-f6c12d2e00b9/volumes"
Nov 04 04:01:55.931342 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:55.931310    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 04:01:55.931752 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:55.931359    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 04:01:55.966152 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:55.966102    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:56.223884 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:56.223801    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:01:56.930726 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:56.930691    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 04:01:56.930899 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:56.930748    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 04:01:57.930575 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:57.930537    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 04:01:57.930976 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:57.930590    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 04:01:58.931064 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:58.931024    6751 patch_prober.go:29] interesting pod/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 04:01:58.931432 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:58.931087    6751 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 04:01:58.944941 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:01:58.944912    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:00.797389 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:00.797361    6751 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:02:00.798378 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:00.798356    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:02:00.799214 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:00.799191    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:02:02.613328 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:02.613272    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:02.846125 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:02.845909    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:03.053753 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:03.053676    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:04.652314 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:04.652280    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:05.038722 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:05.038616    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:05.207927 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:05.207898    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"]
Nov 04 04:02:05.208165 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:05.208142    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy" containerID="cri-o://bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769" gracePeriod=30
Nov 04 04:02:05.210573 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:05.210548    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"]
Nov 04 04:02:05.210750 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:05.210730    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" podUID="a174f8b4-9d90-407f-8914-107ddd97d3db" containerName="istio-proxy" containerID="cri-o://96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069" gracePeriod=30
Nov 04 04:02:05.350456 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:05.350422    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:06.383267 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:06.383233    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:06.598824 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:06.598795    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:06.908003 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:06.907916    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:07.151213 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:07.151183    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:07.376611 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:07.376582    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:07.564949 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:07.564918    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"]
Nov 04 04:02:07.565403 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:07.565187    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy" containerID="cri-o://d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124" gracePeriod=30
Nov 04 04:02:07.570162 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:07.570140    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"]
Nov 04 04:02:07.570345 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:07.570322    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy" containerID="cri-o://6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43" gracePeriod=30
Nov 04 04:02:07.580215 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:07.580188    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"]
Nov 04 04:02:07.580394 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:07.580374    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy" containerID="cri-o://beae82eb790725813b6d8222835e0118e325a7f59ffb3ecfa6be97f6b882aa0d" gracePeriod=30
Nov 04 04:02:07.735285 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:07.725318    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:09.691693 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:09.691658    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:10.465702 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.465677    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 04:02:10.508207 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.508178    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-kube-api-access-dfkmx\" (UniqueName: \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-kube-api-access-dfkmx\") pod \"a174f8b4-9d90-407f-8914-107ddd97d3db\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") "
Nov 04 04:02:10.508355 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.508224    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-envoy\") pod \"a174f8b4-9d90-407f-8914-107ddd97d3db\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") "
Nov 04 04:02:10.508355 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.508279    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a174f8b4-9d90-407f-8914-107ddd97d3db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a174f8b4-9d90-407f-8914-107ddd97d3db-istiod-ca-cert\") pod \"a174f8b4-9d90-407f-8914-107ddd97d3db\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") "
Nov 04 04:02:10.508469 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.508353    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-token\" (UniqueName: \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-token\") pod \"a174f8b4-9d90-407f-8914-107ddd97d3db\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") "
Nov 04 04:02:10.508469 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.508382    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-credential-socket\") pod \"a174f8b4-9d90-407f-8914-107ddd97d3db\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") "
Nov 04 04:02:10.508469 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.508419    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-certs\") pod \"a174f8b4-9d90-407f-8914-107ddd97d3db\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") "
Nov 04 04:02:10.508469 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.508447    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-socket\") pod \"a174f8b4-9d90-407f-8914-107ddd97d3db\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") "
Nov 04 04:02:10.508683 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.508479    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-podinfo\") pod \"a174f8b4-9d90-407f-8914-107ddd97d3db\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") "
Nov 04 04:02:10.508683 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.508553    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-data\") pod \"a174f8b4-9d90-407f-8914-107ddd97d3db\" (UID: \"a174f8b4-9d90-407f-8914-107ddd97d3db\") "
Nov 04 04:02:10.512650 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.509218    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-data" pod "a174f8b4-9d90-407f-8914-107ddd97d3db" (UID: "a174f8b4-9d90-407f-8914-107ddd97d3db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:10.512650 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.509349    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-certs" pod "a174f8b4-9d90-407f-8914-107ddd97d3db" (UID: "a174f8b4-9d90-407f-8914-107ddd97d3db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:10.512650 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.509427    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-socket" pod "a174f8b4-9d90-407f-8914-107ddd97d3db" (UID: "a174f8b4-9d90-407f-8914-107ddd97d3db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:10.512650 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.509528    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a174f8b4-9d90-407f-8914-107ddd97d3db-istiod-ca-cert" pod "a174f8b4-9d90-407f-8914-107ddd97d3db" (UID: "a174f8b4-9d90-407f-8914-107ddd97d3db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:02:10.512650 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.509680    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-credential-socket" pod "a174f8b4-9d90-407f-8914-107ddd97d3db" (UID: "a174f8b4-9d90-407f-8914-107ddd97d3db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:10.512650 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.510986    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-envoy" pod "a174f8b4-9d90-407f-8914-107ddd97d3db" (UID: "a174f8b4-9d90-407f-8914-107ddd97d3db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:10.512650 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.512236    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-token" pod "a174f8b4-9d90-407f-8914-107ddd97d3db" (UID: "a174f8b4-9d90-407f-8914-107ddd97d3db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:10.513079 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.512958    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-kube-api-access-dfkmx" pod "a174f8b4-9d90-407f-8914-107ddd97d3db" (UID: "a174f8b4-9d90-407f-8914-107ddd97d3db"). InnerVolumeSpecName "kube-api-access-dfkmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:10.514306 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.514280    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-podinfo" pod "a174f8b4-9d90-407f-8914-107ddd97d3db" (UID: "a174f8b4-9d90-407f-8914-107ddd97d3db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:02:10.518324 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.518284    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:02:10.610210 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610184    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-podinfo\") pod \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") "
Nov 04 04:02:10.610361 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610239    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istiod-ca-cert\") pod \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") "
Nov 04 04:02:10.610361 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610279    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-credential-socket\") pod \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") "
Nov 04 04:02:10.610361 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610301    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-data\") pod \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") "
Nov 04 04:02:10.610361 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610358    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-kube-api-access-q4tms\" (UniqueName: \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-kube-api-access-q4tms\") pod \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") "
Nov 04 04:02:10.610582 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610389    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-certs\") pod \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") "
Nov 04 04:02:10.610582 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610414    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-token\" (UniqueName: \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-token\") pod \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") "
Nov 04 04:02:10.610582 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610474    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-envoy\") pod \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") "
Nov 04 04:02:10.610582 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610528    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-socket\") pod \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\" (UID: \"74ceb3c9-eee7-4329-ae34-2dc9191b1908\") "
Nov 04 04:02:10.610582 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610564    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-credential-socket" pod "74ceb3c9-eee7-4329-ae34-2dc9191b1908" (UID: "74ceb3c9-eee7-4329-ae34-2dc9191b1908"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:10.610864 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610755    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-data" pod "74ceb3c9-eee7-4329-ae34-2dc9191b1908" (UID: "74ceb3c9-eee7-4329-ae34-2dc9191b1908"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:10.610864 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610680    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istiod-ca-cert" pod "74ceb3c9-eee7-4329-ae34-2dc9191b1908" (UID: "74ceb3c9-eee7-4329-ae34-2dc9191b1908"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:02:10.610864 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610777    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-certs" pod "74ceb3c9-eee7-4329-ae34-2dc9191b1908" (UID: "74ceb3c9-eee7-4329-ae34-2dc9191b1908"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:10.611028 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.610931    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-socket" pod "74ceb3c9-eee7-4329-ae34-2dc9191b1908" (UID: "74ceb3c9-eee7-4329-ae34-2dc9191b1908"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:10.611083 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611067    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611136 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611089    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611136 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611105    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfkmx\" (UniqueName: \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-kube-api-access-dfkmx\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611136 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611121    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611136    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611151    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611166    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611180    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a174f8b4-9d90-407f-8914-107ddd97d3db-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611193    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611208    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611222    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611235    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611248    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a174f8b4-9d90-407f-8914-107ddd97d3db-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.611287 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.611262    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a174f8b4-9d90-407f-8914-107ddd97d3db-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.612419 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.612401    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-podinfo" pod "74ceb3c9-eee7-4329-ae34-2dc9191b1908" (UID: "74ceb3c9-eee7-4329-ae34-2dc9191b1908"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:02:10.612585 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.612567    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-kube-api-access-q4tms" pod "74ceb3c9-eee7-4329-ae34-2dc9191b1908" (UID: "74ceb3c9-eee7-4329-ae34-2dc9191b1908"). InnerVolumeSpecName "kube-api-access-q4tms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:10.613031 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.613009    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-token" pod "74ceb3c9-eee7-4329-ae34-2dc9191b1908" (UID: "74ceb3c9-eee7-4329-ae34-2dc9191b1908"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:10.613078 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.613026    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-envoy" pod "74ceb3c9-eee7-4329-ae34-2dc9191b1908" (UID: "74ceb3c9-eee7-4329-ae34-2dc9191b1908"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:10.712069 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.712038    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.712069 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.712065    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.712401 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.712079    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74ceb3c9-eee7-4329-ae34-2dc9191b1908-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.712401 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.712091    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4tms\" (UniqueName: \"kubernetes.io/projected/74ceb3c9-eee7-4329-ae34-2dc9191b1908-kube-api-access-q4tms\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:10.890883 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.890858    6751 generic.go:356] "Generic (PLEG): container finished" podID="a174f8b4-9d90-407f-8914-107ddd97d3db" containerID="96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069" exitCode=0
Nov 04 04:02:10.890993 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.890906    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"
Nov 04 04:02:10.890993 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.890938    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" event={"ID":"a174f8b4-9d90-407f-8914-107ddd97d3db","Type":"ContainerDied","Data":"96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069"}
Nov 04 04:02:10.890993 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.890989    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs" event={"ID":"a174f8b4-9d90-407f-8914-107ddd97d3db","Type":"ContainerDied","Data":"529d33cdd59b6663023e0d71633314dace0a2ac825d933daf19eb38bcce73d9e"}
Nov 04 04:02:10.891128 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.891019    6751 scope.go:122] "RemoveContainer" containerID="96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069"
Nov 04 04:02:10.900053 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.899079    6751 generic.go:356] "Generic (PLEG): container finished" podID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerID="bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769" exitCode=0
Nov 04 04:02:10.900053 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.899217    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" event={"ID":"74ceb3c9-eee7-4329-ae34-2dc9191b1908","Type":"ContainerDied","Data":"bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769"}
Nov 04 04:02:10.900053 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.899298    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j" event={"ID":"74ceb3c9-eee7-4329-ae34-2dc9191b1908","Type":"ContainerDied","Data":"c986971735fa4c989da106ee485489895c44dc0c53a6d41f8958a92228db7662"}
Nov 04 04:02:10.900053 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.899398    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"
Nov 04 04:02:10.906984 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.906966    6751 scope.go:122] "RemoveContainer" containerID="96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069"
Nov 04 04:02:10.907293 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:02:10.907273    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069\": container with ID starting with 96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069 not found: ID does not exist" containerID="96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069"
Nov 04 04:02:10.907387 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.907299    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069"} err="failed to get container status \"96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069\": rpc error: code = NotFound desc = could not find container \"96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069\": container with ID starting with 96e0e70de61427046aa944276976a777a3e867d0405fdfb622fd698da3ac6069 not found: ID does not exist"
Nov 04 04:02:10.907387 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.907315    6751 scope.go:122] "RemoveContainer" containerID="bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769"
Nov 04 04:02:10.917866 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.917847    6751 scope.go:122] "RemoveContainer" containerID="bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769"
Nov 04 04:02:10.918129 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:02:10.918102    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769\": container with ID starting with bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769 not found: ID does not exist" containerID="bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769"
Nov 04 04:02:10.918173 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.918139    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769"} err="failed to get container status \"bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769\": rpc error: code = NotFound desc = could not find container \"bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769\": container with ID starting with bfaed0f7aa720a1e4ced8e9c58e2d41f583b5a3239cc85265f0449adf50e8769 not found: ID does not exist"
Nov 04 04:02:10.926239 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.926217    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"]
Nov 04 04:02:10.928281 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.928261    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-xtxjs"]
Nov 04 04:02:10.938343 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.938321    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"]
Nov 04 04:02:10.942814 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:10.942796    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-56a44f-openshift-default-584894595f-fgc2j"]
Nov 04 04:02:11.316807 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:11.316778    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:11.529080 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:11.529015    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:11.783858 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:11.783791    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" path="/var/lib/kubelet/pods/74ceb3c9-eee7-4329-ae34-2dc9191b1908/volumes"
Nov 04 04:02:11.784221 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:11.784207    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a174f8b4-9d90-407f-8914-107ddd97d3db" path="/var/lib/kubelet/pods/a174f8b4-9d90-407f-8914-107ddd97d3db/volumes"
Nov 04 04:02:12.697116 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:02:12.697059    6751 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a8de46_9185_43a4_80c5_cfcedf3dd676.slice/crio-conmon-6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5931acd2_4ef3_4a72_b76f_c792ffee8c4a.slice/crio-beae82eb790725813b6d8222835e0118e325a7f59ffb3ecfa6be97f6b882aa0d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4697d051_eb18_4ed0_92c9_408a109a5bf1.slice/crio-d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5931acd2_4ef3_4a72_b76f_c792ffee8c4a.slice/crio-conmon-beae82eb790725813b6d8222835e0118e325a7f59ffb3ecfa6be97f6b882aa0d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ceb3c9_eee7_4329_ae34_2dc9191b1908.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda174f8b4_9d90_407f_8914_107ddd97d3db.slice/crio-529d33cdd59b6663023e0d71633314dace0a2ac825d933daf19eb38bcce73d9e\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4697d051_eb18_4ed0_92c9_408a109a5bf1.slice/crio-conmon-d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ceb3c9_eee7_4329_ae34_2dc9191b1908.slice/crio-c986971735fa4c989da106ee485489895c44dc0c53a6d41f8958a92228db7662\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda174f8b4_9d90_407f_8914_107ddd97d3db.slice\": RecentStats: unable to find data in memory cache]"
Nov 04 04:02:12.832535 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.832513    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:02:12.860365 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.860343    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:02:12.913781 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.913683    6751 generic.go:356] "Generic (PLEG): container finished" podID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerID="beae82eb790725813b6d8222835e0118e325a7f59ffb3ecfa6be97f6b882aa0d" exitCode=0
Nov 04 04:02:12.914062 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.914031    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" event={"ID":"5931acd2-4ef3-4a72-b76f-c792ffee8c4a","Type":"ContainerDied","Data":"beae82eb790725813b6d8222835e0118e325a7f59ffb3ecfa6be97f6b882aa0d"}
Nov 04 04:02:12.914221 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.914113    6751 scope.go:122] "RemoveContainer" containerID="05b636eb769f7a23c2e7b48f04d36d3a24fd15bcc1a8928a27d0719dbceef0b3"
Nov 04 04:02:12.916354 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.915982    6751 generic.go:356] "Generic (PLEG): container finished" podID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerID="6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43" exitCode=0
Nov 04 04:02:12.916354 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.916145    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" event={"ID":"52a8de46-9185-43a4-80c5-cfcedf3dd676","Type":"ContainerDied","Data":"6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43"}
Nov 04 04:02:12.916354 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.916196    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj" event={"ID":"52a8de46-9185-43a4-80c5-cfcedf3dd676","Type":"ContainerDied","Data":"07b955da16452896206b54faed0340f69f4f2411d88563f26c44897b32c4f084"}
Nov 04 04:02:12.916354 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.916315    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"
Nov 04 04:02:12.918305 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.918275    6751 generic.go:356] "Generic (PLEG): container finished" podID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerID="d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124" exitCode=0
Nov 04 04:02:12.918419 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.918324    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" event={"ID":"4697d051-eb18-4ed0-92c9-408a109a5bf1","Type":"ContainerDied","Data":"d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124"}
Nov 04 04:02:12.918419 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.918381    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb" event={"ID":"4697d051-eb18-4ed0-92c9-408a109a5bf1","Type":"ContainerDied","Data":"2b5ef7614848d42c8566f33a3a25bd8e262e1ddb3a8095ea26c69df238c75a3a"}
Nov 04 04:02:12.918419 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.918329    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"
Nov 04 04:02:12.926568 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.926541    6751 scope.go:122] "RemoveContainer" containerID="6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43"
Nov 04 04:02:12.929887 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.929858    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-token\" (UniqueName: \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-token\") pod \"52a8de46-9185-43a4-80c5-cfcedf3dd676\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") "
Nov 04 04:02:12.929984 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.929902    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-data\") pod \"4697d051-eb18-4ed0-92c9-408a109a5bf1\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") "
Nov 04 04:02:12.929984 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.929941    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-podinfo\") pod \"4697d051-eb18-4ed0-92c9-408a109a5bf1\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") "
Nov 04 04:02:12.930066 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.929982    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-envoy\") pod \"4697d051-eb18-4ed0-92c9-408a109a5bf1\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") "
Nov 04 04:02:12.930066 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930012    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-token\" (UniqueName: \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-token\") pod \"4697d051-eb18-4ed0-92c9-408a109a5bf1\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") "
Nov 04 04:02:12.930169 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930081    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-credential-socket\") pod \"52a8de46-9185-43a4-80c5-cfcedf3dd676\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") "
Nov 04 04:02:12.930270 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930202    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-socket\") pod \"4697d051-eb18-4ed0-92c9-408a109a5bf1\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") "
Nov 04 04:02:12.930270 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930241    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-envoy\") pod \"52a8de46-9185-43a4-80c5-cfcedf3dd676\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") "
Nov 04 04:02:12.930387 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930295    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52a8de46-9185-43a4-80c5-cfcedf3dd676-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52a8de46-9185-43a4-80c5-cfcedf3dd676-istiod-ca-cert\") pod \"52a8de46-9185-43a4-80c5-cfcedf3dd676\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") "
Nov 04 04:02:12.930387 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930338    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-data\") pod \"52a8de46-9185-43a4-80c5-cfcedf3dd676\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") "
Nov 04 04:02:12.930387 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930370    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-kube-api-access-f9clr\" (UniqueName: \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-kube-api-access-f9clr\") pod \"52a8de46-9185-43a4-80c5-cfcedf3dd676\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") "
Nov 04 04:02:12.930536 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930421    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-certs\") pod \"4697d051-eb18-4ed0-92c9-408a109a5bf1\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") "
Nov 04 04:02:12.930536 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930441    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-data" pod "4697d051-eb18-4ed0-92c9-408a109a5bf1" (UID: "4697d051-eb18-4ed0-92c9-408a109a5bf1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:12.930536 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930460    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-podinfo\") pod \"52a8de46-9185-43a4-80c5-cfcedf3dd676\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") "
Nov 04 04:02:12.930536 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930490    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-certs\") pod \"52a8de46-9185-43a4-80c5-cfcedf3dd676\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") "
Nov 04 04:02:12.930536 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930532    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4697d051-eb18-4ed0-92c9-408a109a5bf1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4697d051-eb18-4ed0-92c9-408a109a5bf1-istiod-ca-cert\") pod \"4697d051-eb18-4ed0-92c9-408a109a5bf1\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") "
Nov 04 04:02:12.930807 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930578    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-credential-socket\") pod \"4697d051-eb18-4ed0-92c9-408a109a5bf1\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") "
Nov 04 04:02:12.930807 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930607    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-socket\") pod \"52a8de46-9185-43a4-80c5-cfcedf3dd676\" (UID: \"52a8de46-9185-43a4-80c5-cfcedf3dd676\") "
Nov 04 04:02:12.930807 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930665    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-kube-api-access-plj2r\" (UniqueName: \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-kube-api-access-plj2r\") pod \"4697d051-eb18-4ed0-92c9-408a109a5bf1\" (UID: \"4697d051-eb18-4ed0-92c9-408a109a5bf1\") "
Nov 04 04:02:12.930965 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930851    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a8de46-9185-43a4-80c5-cfcedf3dd676-istiod-ca-cert" pod "52a8de46-9185-43a4-80c5-cfcedf3dd676" (UID: "52a8de46-9185-43a4-80c5-cfcedf3dd676"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.931076    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-certs" pod "4697d051-eb18-4ed0-92c9-408a109a5bf1" (UID: "4697d051-eb18-4ed0-92c9-408a109a5bf1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.931226    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.931247    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52a8de46-9185-43a4-80c5-cfcedf3dd676-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.931262    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.930905    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-certs" pod "52a8de46-9185-43a4-80c5-cfcedf3dd676" (UID: "52a8de46-9185-43a4-80c5-cfcedf3dd676"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.931490    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-credential-socket" pod "4697d051-eb18-4ed0-92c9-408a109a5bf1" (UID: "4697d051-eb18-4ed0-92c9-408a109a5bf1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.931522    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-socket" pod "4697d051-eb18-4ed0-92c9-408a109a5bf1" (UID: "4697d051-eb18-4ed0-92c9-408a109a5bf1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.931655    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-credential-socket" pod "52a8de46-9185-43a4-80c5-cfcedf3dd676" (UID: "52a8de46-9185-43a4-80c5-cfcedf3dd676"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.931751    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-socket" pod "52a8de46-9185-43a4-80c5-cfcedf3dd676" (UID: "52a8de46-9185-43a4-80c5-cfcedf3dd676"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.931863    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-data" pod "52a8de46-9185-43a4-80c5-cfcedf3dd676" (UID: "52a8de46-9185-43a4-80c5-cfcedf3dd676"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:12.931888 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.931859    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4697d051-eb18-4ed0-92c9-408a109a5bf1-istiod-ca-cert" pod "4697d051-eb18-4ed0-92c9-408a109a5bf1" (UID: "4697d051-eb18-4ed0-92c9-408a109a5bf1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:02:12.933350 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.933322    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-kube-api-access-plj2r" pod "4697d051-eb18-4ed0-92c9-408a109a5bf1" (UID: "4697d051-eb18-4ed0-92c9-408a109a5bf1"). InnerVolumeSpecName "kube-api-access-plj2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:12.933697 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.933672    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-podinfo" pod "52a8de46-9185-43a4-80c5-cfcedf3dd676" (UID: "52a8de46-9185-43a4-80c5-cfcedf3dd676"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:02:12.933970 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.933946    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-envoy" pod "52a8de46-9185-43a4-80c5-cfcedf3dd676" (UID: "52a8de46-9185-43a4-80c5-cfcedf3dd676"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:12.935009 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.934983    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-token" pod "52a8de46-9185-43a4-80c5-cfcedf3dd676" (UID: "52a8de46-9185-43a4-80c5-cfcedf3dd676"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:12.935009 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.934990    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-token" pod "4697d051-eb18-4ed0-92c9-408a109a5bf1" (UID: "4697d051-eb18-4ed0-92c9-408a109a5bf1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:12.935233 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.935211    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-envoy" pod "4697d051-eb18-4ed0-92c9-408a109a5bf1" (UID: "4697d051-eb18-4ed0-92c9-408a109a5bf1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:12.935482 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.935454    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-podinfo" pod "4697d051-eb18-4ed0-92c9-408a109a5bf1" (UID: "4697d051-eb18-4ed0-92c9-408a109a5bf1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:02:12.936348 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.936317    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-kube-api-access-f9clr" pod "52a8de46-9185-43a4-80c5-cfcedf3dd676" (UID: "52a8de46-9185-43a4-80c5-cfcedf3dd676"). InnerVolumeSpecName "kube-api-access-f9clr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:12.936933 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.936912    6751 scope.go:122] "RemoveContainer" containerID="6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43"
Nov 04 04:02:12.937220 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:02:12.937200    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43\": container with ID starting with 6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43 not found: ID does not exist" containerID="6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43"
Nov 04 04:02:12.937270 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.937231    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43"} err="failed to get container status \"6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43\": rpc error: code = NotFound desc = could not find container \"6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43\": container with ID starting with 6bf86dfccd6539120da827b62ebf094791d95094f10745f0370a212b99b77c43 not found: ID does not exist"
Nov 04 04:02:12.937270 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.937247    6751 scope.go:122] "RemoveContainer" containerID="d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124"
Nov 04 04:02:12.948352 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.948331    6751 scope.go:122] "RemoveContainer" containerID="d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124"
Nov 04 04:02:12.948837 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:02:12.948809    6751 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124\": container with ID starting with d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124 not found: ID does not exist" containerID="d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124"
Nov 04 04:02:12.948939 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:12.948847    6751 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124"} err="failed to get container status \"d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124\": rpc error: code = NotFound desc = could not find container \"d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124\": container with ID starting with d380ec444bb129ae4f3d3aaabf4121c07f09401f0dd926b6a7fc6d084b648124 not found: ID does not exist"
Nov 04 04:02:13.032839 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032807    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.032839 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032834    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.032839 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032844    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4697d051-eb18-4ed0-92c9-408a109a5bf1-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032853    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032862    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032874    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plj2r\" (UniqueName: \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-kube-api-access-plj2r\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032887    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032896    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032905    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032914    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4697d051-eb18-4ed0-92c9-408a109a5bf1-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032921    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032930    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4697d051-eb18-4ed0-92c9-408a109a5bf1-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032937    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032948    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52a8de46-9185-43a4-80c5-cfcedf3dd676-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.033065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.032961    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9clr\" (UniqueName: \"kubernetes.io/projected/52a8de46-9185-43a4-80c5-cfcedf3dd676-kube-api-access-f9clr\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.217441 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.217420    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 04:02:13.279148 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.279118    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"]
Nov 04 04:02:13.293319 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.293289    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-hp7nb"]
Nov 04 04:02:13.312042 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.312016    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"]
Nov 04 04:02:13.315436 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.315414    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-jfwfj"]
Nov 04 04:02:13.336142 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336119    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istiod-ca-cert\") pod \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") "
Nov 04 04:02:13.336261 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336172    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-podinfo\") pod \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") "
Nov 04 04:02:13.336261 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336223    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-socket\") pod \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") "
Nov 04 04:02:13.336261 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336256    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-data\") pod \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") "
Nov 04 04:02:13.336420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336286    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-credential-socket\") pod \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") "
Nov 04 04:02:13.336420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336331    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-token\" (UniqueName: \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-token\") pod \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") "
Nov 04 04:02:13.336420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336397    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-envoy\") pod \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") "
Nov 04 04:02:13.336566 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336428    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-certs\") pod \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") "
Nov 04 04:02:13.336566 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336459    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-kube-api-access-cdjl9\" (UniqueName: \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-kube-api-access-cdjl9\") pod \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\" (UID: \"5931acd2-4ef3-4a72-b76f-c792ffee8c4a\") "
Nov 04 04:02:13.336566 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336548    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-data" pod "5931acd2-4ef3-4a72-b76f-c792ffee8c4a" (UID: "5931acd2-4ef3-4a72-b76f-c792ffee8c4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:13.336741 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336583    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istiod-ca-cert" pod "5931acd2-4ef3-4a72-b76f-c792ffee8c4a" (UID: "5931acd2-4ef3-4a72-b76f-c792ffee8c4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:02:13.336741 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336558    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-socket" pod "5931acd2-4ef3-4a72-b76f-c792ffee8c4a" (UID: "5931acd2-4ef3-4a72-b76f-c792ffee8c4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:13.336844 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336747    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-certs" pod "5931acd2-4ef3-4a72-b76f-c792ffee8c4a" (UID: "5931acd2-4ef3-4a72-b76f-c792ffee8c4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:13.336940 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.336914    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-credential-socket" pod "5931acd2-4ef3-4a72-b76f-c792ffee8c4a" (UID: "5931acd2-4ef3-4a72-b76f-c792ffee8c4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:13.337052 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.337019    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.337052 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.337038    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.337052 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.337048    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.337179 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.337062    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.337179 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.337074    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.338893 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.338874    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-envoy" pod "5931acd2-4ef3-4a72-b76f-c792ffee8c4a" (UID: "5931acd2-4ef3-4a72-b76f-c792ffee8c4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:13.338893 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.338882    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-podinfo" pod "5931acd2-4ef3-4a72-b76f-c792ffee8c4a" (UID: "5931acd2-4ef3-4a72-b76f-c792ffee8c4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:02:13.339093 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.339071    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-token" pod "5931acd2-4ef3-4a72-b76f-c792ffee8c4a" (UID: "5931acd2-4ef3-4a72-b76f-c792ffee8c4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:13.339163 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.339087    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-kube-api-access-cdjl9" pod "5931acd2-4ef3-4a72-b76f-c792ffee8c4a" (UID: "5931acd2-4ef3-4a72-b76f-c792ffee8c4a"). InnerVolumeSpecName "kube-api-access-cdjl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:13.438686 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.438596    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.438686 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.438665    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.438686 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.438684    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.438889 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.438699    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdjl9\" (UniqueName: \"kubernetes.io/projected/5931acd2-4ef3-4a72-b76f-c792ffee8c4a-kube-api-access-cdjl9\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:13.460446 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.460411    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:13.685369 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.685335    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:13.796322 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.796243    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" path="/var/lib/kubelet/pods/4697d051-eb18-4ed0-92c9-408a109a5bf1/volumes"
Nov 04 04:02:13.796728 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.796711    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" path="/var/lib/kubelet/pods/52a8de46-9185-43a4-80c5-cfcedf3dd676/volumes"
Nov 04 04:02:13.911889 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.911842    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:13.927210 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.927174    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn" event={"ID":"5931acd2-4ef3-4a72-b76f-c792ffee8c4a","Type":"ContainerDied","Data":"1cf5a3b7597fb97d5ffe939031d00c2edd2f518ccccd26fbbced635e04075c74"}
Nov 04 04:02:13.927339 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.927212    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"
Nov 04 04:02:13.927339 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.927268    6751 scope.go:122] "RemoveContainer" containerID="beae82eb790725813b6d8222835e0118e325a7f59ffb3ecfa6be97f6b882aa0d"
Nov 04 04:02:13.950586 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.950558    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"]
Nov 04 04:02:13.955872 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:13.955850    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-64c20d-openshift-default-d97b6c887-fxfhn"]
Nov 04 04:02:15.190459 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:15.190423    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:15.595333 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:15.595295    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:15.792660 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:15.787061    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" path="/var/lib/kubelet/pods/5931acd2-4ef3-4a72-b76f-c792ffee8c4a/volumes"
Nov 04 04:02:15.825314 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:15.825283    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"]
Nov 04 04:02:15.825531 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:15.825509    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy" containerID="cri-o://7c7d40671b21d414f3da72fca474ae3628f154a8dc6500c2c618c1e6c0a6ba8e" gracePeriod=30
Nov 04 04:02:15.827236 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:15.827213    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"]
Nov 04 04:02:15.827403 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:15.827385    6751 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy" containerID="cri-o://80ac47283b18d293f6184a2d5b21156ffa625c313c421490cfba2dbb208b893c" gracePeriod=30
Nov 04 04:02:15.998020 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:15.997935    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:17.630165 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:17.630133    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:17.864444 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:17.864418    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:20.945367 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:20.945343    6751 patch_prober.go:29] interesting pod/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Nov 04 04:02:20.945709 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:20.945402    6751 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Nov 04 04:02:20.974885 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:20.974852    6751 generic.go:356] "Generic (PLEG): container finished" podID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerID="7c7d40671b21d414f3da72fca474ae3628f154a8dc6500c2c618c1e6c0a6ba8e" exitCode=0
Nov 04 04:02:20.975065 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:20.975025    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" event={"ID":"f8e1215b-ab44-4926-81e7-dcb61def4ad5","Type":"ContainerDied","Data":"7c7d40671b21d414f3da72fca474ae3628f154a8dc6500c2c618c1e6c0a6ba8e"}
Nov 04 04:02:20.975149 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:20.975088    6751 scope.go:122] "RemoveContainer" containerID="788e4f9880730e1022bacb655aff628f35a78e52c979213aa4321bf9c74a6eb5"
Nov 04 04:02:20.977183 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:20.977156    6751 generic.go:356] "Generic (PLEG): container finished" podID="56656b54-8f49-4a46-92a6-59400b873509" containerID="80ac47283b18d293f6184a2d5b21156ffa625c313c421490cfba2dbb208b893c" exitCode=0
Nov 04 04:02:20.977277 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:20.977194    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" event={"ID":"56656b54-8f49-4a46-92a6-59400b873509","Type":"ContainerDied","Data":"80ac47283b18d293f6184a2d5b21156ffa625c313c421490cfba2dbb208b893c"}
Nov 04 04:02:21.088037 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:02:21.088006    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 04:02:21.104883 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.104819    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 04:02:21.106960 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.106941    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:02:21.232687 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.232658    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-podinfo\") pod \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") "
Nov 04 04:02:21.232848 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.232691    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-data\") pod \"56656b54-8f49-4a46-92a6-59400b873509\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") "
Nov 04 04:02:21.232848 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.232719    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-envoy\") pod \"56656b54-8f49-4a46-92a6-59400b873509\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") "
Nov 04 04:02:21.232848 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.232749    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istiod-ca-cert\") pod \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") "
Nov 04 04:02:21.232848 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.232773    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-istio-token\" (UniqueName: \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-istio-token\") pod \"56656b54-8f49-4a46-92a6-59400b873509\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") "
Nov 04 04:02:21.233068 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.232846    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-credential-socket\") pod \"56656b54-8f49-4a46-92a6-59400b873509\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") "
Nov 04 04:02:21.233068 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.232943    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-token\" (UniqueName: \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-token\") pod \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") "
Nov 04 04:02:21.233068 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.232993    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-envoy\") pod \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") "
Nov 04 04:02:21.233068 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233013    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-data" pod "56656b54-8f49-4a46-92a6-59400b873509" (UID: "56656b54-8f49-4a46-92a6-59400b873509"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:21.233068 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233031    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-socket\") pod \"56656b54-8f49-4a46-92a6-59400b873509\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") "
Nov 04 04:02:21.233309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233070    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-credential-socket" pod "56656b54-8f49-4a46-92a6-59400b873509" (UID: "56656b54-8f49-4a46-92a6-59400b873509"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:21.233309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233134    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-credential-socket\") pod \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") "
Nov 04 04:02:21.233309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233164    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-kube-api-access-2k2lt\" (UniqueName: \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-kube-api-access-2k2lt\") pod \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") "
Nov 04 04:02:21.233309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233185    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-data\") pod \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") "
Nov 04 04:02:21.233309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233202    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/56656b54-8f49-4a46-92a6-59400b873509-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56656b54-8f49-4a46-92a6-59400b873509-istio-podinfo\") pod \"56656b54-8f49-4a46-92a6-59400b873509\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") "
Nov 04 04:02:21.233309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233211    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istiod-ca-cert" pod "f8e1215b-ab44-4926-81e7-dcb61def4ad5" (UID: "f8e1215b-ab44-4926-81e7-dcb61def4ad5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:02:21.233309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233229    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-socket\") pod \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") "
Nov 04 04:02:21.233309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233250    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-kube-api-access-x69bf\" (UniqueName: \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-kube-api-access-x69bf\") pod \"56656b54-8f49-4a46-92a6-59400b873509\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") "
Nov 04 04:02:21.233309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233287    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-certs\") pod \"56656b54-8f49-4a46-92a6-59400b873509\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") "
Nov 04 04:02:21.233309 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233313    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56656b54-8f49-4a46-92a6-59400b873509-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56656b54-8f49-4a46-92a6-59400b873509-istiod-ca-cert\") pod \"56656b54-8f49-4a46-92a6-59400b873509\" (UID: \"56656b54-8f49-4a46-92a6-59400b873509\") "
Nov 04 04:02:21.233863 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233376    6751 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-certs\") pod \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\" (UID: \"f8e1215b-ab44-4926-81e7-dcb61def4ad5\") "
Nov 04 04:02:21.233863 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233740    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-socket" pod "56656b54-8f49-4a46-92a6-59400b873509" (UID: "56656b54-8f49-4a46-92a6-59400b873509"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:21.233863 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233778    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.233863 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233802    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.233863 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233820    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.234129 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.233932    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-socket" pod "f8e1215b-ab44-4926-81e7-dcb61def4ad5" (UID: "f8e1215b-ab44-4926-81e7-dcb61def4ad5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:21.234129 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.234027    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-certs" pod "f8e1215b-ab44-4926-81e7-dcb61def4ad5" (UID: "f8e1215b-ab44-4926-81e7-dcb61def4ad5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:21.234236 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.234140    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-data" pod "f8e1215b-ab44-4926-81e7-dcb61def4ad5" (UID: "f8e1215b-ab44-4926-81e7-dcb61def4ad5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:21.235458 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.235433    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-podinfo" pod "f8e1215b-ab44-4926-81e7-dcb61def4ad5" (UID: "f8e1215b-ab44-4926-81e7-dcb61def4ad5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:02:21.235584 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.235554    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-envoy" pod "f8e1215b-ab44-4926-81e7-dcb61def4ad5" (UID: "f8e1215b-ab44-4926-81e7-dcb61def4ad5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:21.235704 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.235676    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-certs" pod "56656b54-8f49-4a46-92a6-59400b873509" (UID: "56656b54-8f49-4a46-92a6-59400b873509"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:21.235803 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.235779    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-kube-api-access-2k2lt" pod "f8e1215b-ab44-4926-81e7-dcb61def4ad5" (UID: "f8e1215b-ab44-4926-81e7-dcb61def4ad5"). InnerVolumeSpecName "kube-api-access-2k2lt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:21.235911 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.235886    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-credential-socket" pod "f8e1215b-ab44-4926-81e7-dcb61def4ad5" (UID: "f8e1215b-ab44-4926-81e7-dcb61def4ad5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:21.236036 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.235918    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56656b54-8f49-4a46-92a6-59400b873509-istiod-ca-cert" pod "56656b54-8f49-4a46-92a6-59400b873509" (UID: "56656b54-8f49-4a46-92a6-59400b873509"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 04:02:21.236324 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.236292    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-token" pod "f8e1215b-ab44-4926-81e7-dcb61def4ad5" (UID: "f8e1215b-ab44-4926-81e7-dcb61def4ad5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:21.236386 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.236338    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-kube-api-access-x69bf" pod "56656b54-8f49-4a46-92a6-59400b873509" (UID: "56656b54-8f49-4a46-92a6-59400b873509"). InnerVolumeSpecName "kube-api-access-x69bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:21.236472 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.236453    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-envoy" pod "56656b54-8f49-4a46-92a6-59400b873509" (UID: "56656b54-8f49-4a46-92a6-59400b873509"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 04:02:21.236567 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.236552    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-istio-token" pod "56656b54-8f49-4a46-92a6-59400b873509" (UID: "56656b54-8f49-4a46-92a6-59400b873509"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 04:02:21.236726 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.236707    6751 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56656b54-8f49-4a46-92a6-59400b873509-istio-podinfo" pod "56656b54-8f49-4a46-92a6-59400b873509" (UID: "56656b54-8f49-4a46-92a6-59400b873509"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 04:02:21.334844 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334815    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334844 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334839    6751 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56656b54-8f49-4a46-92a6-59400b873509-istiod-ca-cert\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334848    6751 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-certs\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334858    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334867    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334875    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334884    6751 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-token\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334891    6751 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-envoy\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334900    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56656b54-8f49-4a46-92a6-59400b873509-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334908    6751 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-credential-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334917    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k2lt\" (UniqueName: \"kubernetes.io/projected/f8e1215b-ab44-4926-81e7-dcb61def4ad5-kube-api-access-2k2lt\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334926    6751 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-istio-data\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334934    6751 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56656b54-8f49-4a46-92a6-59400b873509-istio-podinfo\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334942    6751 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e1215b-ab44-4926-81e7-dcb61def4ad5-workload-socket\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.334983 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.334950    6751 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x69bf\" (UniqueName: \"kubernetes.io/projected/56656b54-8f49-4a46-92a6-59400b873509-kube-api-access-x69bf\") on node \"ip-10-0-129-253.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 04:02:21.557341 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.557278    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:21.765196 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.765163    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:21.983403 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.983375    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"
Nov 04 04:02:21.983835 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.983422    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx" event={"ID":"f8e1215b-ab44-4926-81e7-dcb61def4ad5","Type":"ContainerDied","Data":"013bb66c380b60c7f2513dd89265d2290919863a3d2c08409d0a4c7bc0d64f54"}
Nov 04 04:02:21.983835 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.983471    6751 scope.go:122] "RemoveContainer" containerID="7c7d40671b21d414f3da72fca474ae3628f154a8dc6500c2c618c1e6c0a6ba8e"
Nov 04 04:02:21.984889 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.984860    6751 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"
Nov 04 04:02:21.984889 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.984869    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2" event={"ID":"56656b54-8f49-4a46-92a6-59400b873509","Type":"ContainerDied","Data":"f1bbff2e323241bae01b378a2621ce3cadf2d65ecf63926e69e3d1f4a9d2931e"}
Nov 04 04:02:21.992510 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:21.992491    6751 scope.go:122] "RemoveContainer" containerID="80ac47283b18d293f6184a2d5b21156ffa625c313c421490cfba2dbb208b893c"
Nov 04 04:02:22.004725 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:22.004703    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"]
Nov 04 04:02:22.010749 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:22.010727    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-6tdsx"]
Nov 04 04:02:22.027615 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:22.027593    6751 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"]
Nov 04 04:02:22.032827 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:22.032807    6751 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4afe0e-openshift-default-5f76c4b57f-cwvw2"]
Nov 04 04:02:23.784193 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:23.784160    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56656b54-8f49-4a46-92a6-59400b873509" path="/var/lib/kubelet/pods/56656b54-8f49-4a46-92a6-59400b873509/volumes"
Nov 04 04:02:23.784580 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:23.784569    6751 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" path="/var/lib/kubelet/pods/f8e1215b-ab44-4926-81e7-dcb61def4ad5/volumes"
Nov 04 04:02:27.436678 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:27.436644    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:27.785748 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:27.785676    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:02:28.881780 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:28.881748    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-79b649855b-9p994_1636e316-ede4-4230-a002-46b1fcea8469/manager/0.log"
Nov 04 04:02:31.411330 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:31.411301    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0539c1b2-tlmfj_9073d5de-a959-496e-94e0-4ca6ecf2d37c/revoke-keys/0.log"
Nov 04 04:02:31.580206 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:31.580177    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0619452a-6tk5j_e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc/revoke-keys/0.log"
Nov 04 04:02:31.751750 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:31.751686    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-28adf67f-rnb9l_31bba333-47fa-431a-b65b-eaf49669c567/revoke-keys/0.log"
Nov 04 04:02:31.914928 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:31.914902    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-2a982a8a-znmgm_63ef4111-7492-4f9a-99aa-c1e66c451488/revoke-keys/0.log"
Nov 04 04:02:32.079564 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:32.079501    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-464e2bd6-fmfxd_59ace1c6-1907-424a-a071-5d7fd00aea0d/revoke-keys/0.log"
Nov 04 04:02:32.232268 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:32.232247    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-5cccef37-hxdpd_acc5759a-fd5e-4e1a-94ef-57a90e9806a9/revoke-keys/0.log"
Nov 04 04:02:32.395820 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:32.395801    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-66f93416-k5vfs_4381712a-9961-4bc9-8b53-be50182422c5/revoke-keys/0.log"
Nov 04 04:02:32.550528 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:32.550504    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-8f335aa5-4x5dt_eaaf9a0f-0904-4722-80b4-a36d7bd4192d/revoke-keys/0.log"
Nov 04 04:02:32.701873 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:32.701820    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-914d691e-sxlhv_9c0ad55b-07ca-43c4-ac54-870f2baa82b0/revoke-keys/0.log"
Nov 04 04:02:32.855047 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:32.855015    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-92bab82f-nvbrz_06128bb3-f48f-4901-8c9e-4d1f0e24cb1f/revoke-keys/0.log"
Nov 04 04:02:33.011521 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:33.011466    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-cc076113-rqmv8_f093ec8c-4edd-48d9-aace-3139fa885014/revoke-keys/0.log"
Nov 04 04:02:33.185939 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:33.185919    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-cd9fe1a4-88lrc_62c7a280-ae91-4bca-8370-5e1d88cb9745/revoke-keys/0.log"
Nov 04 04:02:33.391945 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:33.391922    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-d1017559-g47dg_19163451-75a0-4fe0-a8c4-97c280b2ebd7/revoke-keys/0.log"
Nov 04 04:02:33.572928 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:33.572908    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-ec7bdfdd-4drjl_ef77f1d6-3638-4e63-ad6c-b0e8ea320df2/revoke-keys/0.log"
Nov 04 04:02:38.118925 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:38.118893    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-82hvc_b030923a-85f1-432c-accd-ac50ab54f8ee/discovery/0.log"
Nov 04 04:02:38.319738 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:38.319711    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bd469f65f-4x7zx_fc13f463-fe9a-4482-9d0a-fac5a2b845bb/kube-auth-proxy/0.log"
Nov 04 04:02:38.860111 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:38.860084    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-96lb9_13ffed84-554f-440a-8016-20e8344e5135/istio-proxy/0.log"
Nov 04 04:02:39.765706 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:39.765674    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-wjbj2_7427f8fe-4e90-426b-9e23-7054641123b3/istio-proxy/0.log"
Nov 04 04:02:39.940495 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:39.940469    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-xrqpp_90da3418-09f9-4d87-b646-75914ed6a3bc/istio-proxy/0.log"
Nov 04 04:02:42.103104 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:42.103075    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-l5tsl_21a18d39-f8f7-4cc8-a946-312e07cfce01/main/0.log"
Nov 04 04:02:42.442697 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:42.442614    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2_ecc0ca03-4870-4c00-a872-f2d0928bd23c/main/0.log"
Nov 04 04:02:42.619292 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:42.619251    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-zn2mw_2dcaadca-0d52-4448-a2ad-51788bbeaa80/main/0.log"
Nov 04 04:02:42.968793 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:42.968765    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq_1669eab8-e98a-4eb5-838d-943fb4ba5d2b/main/0.log"
Nov 04 04:02:51.086403 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:02:51.086361    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 04:02:52.959635 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:52.959595    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-79b649855b-9p994_1636e316-ede4-4230-a002-46b1fcea8469/manager/0.log"
Nov 04 04:02:54.983577 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:54.983541    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0539c1b2-tlmfj_9073d5de-a959-496e-94e0-4ca6ecf2d37c/revoke-keys/0.log"
Nov 04 04:02:55.155318 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:55.155289    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0619452a-6tk5j_e1d4f2ef-c1eb-4212-9c39-2dfe538d4fbc/revoke-keys/0.log"
Nov 04 04:02:55.319535 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:55.319450    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-28adf67f-rnb9l_31bba333-47fa-431a-b65b-eaf49669c567/revoke-keys/0.log"
Nov 04 04:02:55.499358 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:55.499327    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-2a982a8a-znmgm_63ef4111-7492-4f9a-99aa-c1e66c451488/revoke-keys/0.log"
Nov 04 04:02:55.656461 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:55.656431    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-464e2bd6-fmfxd_59ace1c6-1907-424a-a071-5d7fd00aea0d/revoke-keys/0.log"
Nov 04 04:02:55.818289 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:55.818265    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-5cccef37-hxdpd_acc5759a-fd5e-4e1a-94ef-57a90e9806a9/revoke-keys/0.log"
Nov 04 04:02:55.993012 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:55.992941    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-66f93416-k5vfs_4381712a-9961-4bc9-8b53-be50182422c5/revoke-keys/0.log"
Nov 04 04:02:56.161882 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:56.161858    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-8f335aa5-4x5dt_eaaf9a0f-0904-4722-80b4-a36d7bd4192d/revoke-keys/0.log"
Nov 04 04:02:56.322912 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:56.322829    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-914d691e-sxlhv_9c0ad55b-07ca-43c4-ac54-870f2baa82b0/revoke-keys/0.log"
Nov 04 04:02:56.498204 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:56.498175    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-92bab82f-nvbrz_06128bb3-f48f-4901-8c9e-4d1f0e24cb1f/revoke-keys/0.log"
Nov 04 04:02:56.654037 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:56.654017    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-cc076113-rqmv8_f093ec8c-4edd-48d9-aace-3139fa885014/revoke-keys/0.log"
Nov 04 04:02:56.825310 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:56.825288    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-cd9fe1a4-88lrc_62c7a280-ae91-4bca-8370-5e1d88cb9745/revoke-keys/0.log"
Nov 04 04:02:56.990826 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:56.990764    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-d1017559-g47dg_19163451-75a0-4fe0-a8c4-97c280b2ebd7/revoke-keys/0.log"
Nov 04 04:02:57.153063 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:02:57.153043    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-ec7bdfdd-4drjl_ef77f1d6-3638-4e63-ad6c-b0e8ea320df2/revoke-keys/0.log"
Nov 04 04:03:00.409657 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:00.409611    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-82hvc_b030923a-85f1-432c-accd-ac50ab54f8ee/discovery/0.log"
Nov 04 04:03:00.589993 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:00.589969    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bd469f65f-4x7zx_fc13f463-fe9a-4482-9d0a-fac5a2b845bb/kube-auth-proxy/0.log"
Nov 04 04:03:01.142710 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:01.142685    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-96lb9_13ffed84-554f-440a-8016-20e8344e5135/istio-proxy/0.log"
Nov 04 04:03:02.057957 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:02.057931    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-wjbj2_7427f8fe-4e90-426b-9e23-7054641123b3/istio-proxy/0.log"
Nov 04 04:03:02.238664 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:02.238631    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-xrqpp_90da3418-09f9-4d87-b646-75914ed6a3bc/istio-proxy/0.log"
Nov 04 04:03:03.754947 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:03.754918    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-l5tsl_21a18d39-f8f7-4cc8-a946-312e07cfce01/main/0.log"
Nov 04 04:03:04.102569 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:04.102543    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8djft2_ecc0ca03-4870-4c00-a872-f2d0928bd23c/main/0.log"
Nov 04 04:03:04.281449 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:04.281419    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-zn2mw_2dcaadca-0d52-4448-a2ad-51788bbeaa80/main/0.log"
Nov 04 04:03:04.635277 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:04.635237    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-42mcq_1669eab8-e98a-4eb5-838d-943fb4ba5d2b/main/0.log"
Nov 04 04:03:06.971140 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:06.971105    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:03:21.086298 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:03:21.086263    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 04:03:50.403337 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:03:50.403263    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:03:51.085943 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:03:51.085908    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 04:04:00.157957 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:00.157920    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:04:00.377303 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:00.377274    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:04:03.577048 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:03.577018    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ksb9x_e38ea962-08ee-44b8-8872-feec06d36ada/global-pull-secret-syncer/0.log"
Nov 04 04:04:03.803449 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:03.803423    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log"
Nov 04 04:04:03.932953 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:03.932927    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-253.us-east-2.compute.internal_f909644ddda3f20f4057bfd2523b2509/haproxy/0.log"
Nov 04 04:04:07.017128 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:07.017095    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:04:12.309743 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.309710    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5453641c-2d52-4eff-90d2-4020db02c44c/alertmanager/0.log"
Nov 04 04:04:12.338270 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.338241    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5453641c-2d52-4eff-90d2-4020db02c44c/config-reloader/0.log"
Nov 04 04:04:12.367872 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.367850    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5453641c-2d52-4eff-90d2-4020db02c44c/kube-rbac-proxy-web/0.log"
Nov 04 04:04:12.401790 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.401772    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5453641c-2d52-4eff-90d2-4020db02c44c/kube-rbac-proxy/0.log"
Nov 04 04:04:12.449495 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.449477    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5453641c-2d52-4eff-90d2-4020db02c44c/kube-rbac-proxy-metric/0.log"
Nov 04 04:04:12.478352 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.478333    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5453641c-2d52-4eff-90d2-4020db02c44c/prom-label-proxy/0.log"
Nov 04 04:04:12.509610 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.509592    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5453641c-2d52-4eff-90d2-4020db02c44c/init-config-reloader/0.log"
Nov 04 04:04:12.596628 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.596604    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-bgjtc_aae100a6-d602-4c8b-b301-2f7a7772fe77/kube-state-metrics/0.log"
Nov 04 04:04:12.621392 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.621373    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-bgjtc_aae100a6-d602-4c8b-b301-2f7a7772fe77/kube-rbac-proxy-main/0.log"
Nov 04 04:04:12.653847 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.653828    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-bgjtc_aae100a6-d602-4c8b-b301-2f7a7772fe77/kube-rbac-proxy-self/0.log"
Nov 04 04:04:12.687454 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.687433    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5558d76459-nzh9f_9b84a44a-cb54-47a0-834b-d5794c8d59ad/metrics-server/0.log"
Nov 04 04:04:12.874814 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.874724    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nxhrm_74f52bd8-5314-4ba0-8834-81e86a91c1f6/node-exporter/0.log"
Nov 04 04:04:12.906587 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.906568    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nxhrm_74f52bd8-5314-4ba0-8834-81e86a91c1f6/kube-rbac-proxy/0.log"
Nov 04 04:04:12.934865 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:12.934847    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nxhrm_74f52bd8-5314-4ba0-8834-81e86a91c1f6/init-textfile/0.log"
Nov 04 04:04:13.081121 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.081100    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-nmfcl_34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/kube-rbac-proxy-main/0.log"
Nov 04 04:04:13.110101 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.110083    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-nmfcl_34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/kube-rbac-proxy-self/0.log"
Nov 04 04:04:13.138286 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.138220    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-nmfcl_34bf14b5-d08b-4a4f-af25-2e24d53fdf4d/openshift-state-metrics/0.log"
Nov 04 04:04:13.513707 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.513573    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f6fc89b6-7k47b_bd169cd6-f080-4fc4-82b4-cdb781a09640/telemeter-client/0.log"
Nov 04 04:04:13.542249 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.542226    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f6fc89b6-7k47b_bd169cd6-f080-4fc4-82b4-cdb781a09640/reload/0.log"
Nov 04 04:04:13.568730 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.568708    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f6fc89b6-7k47b_bd169cd6-f080-4fc4-82b4-cdb781a09640/kube-rbac-proxy/0.log"
Nov 04 04:04:13.724147 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724119    6751 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"]
Nov 04 04:04:13.724736 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724716    6751 cpu_manager.go:414] "containerMap: removing container" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy"
Nov 04 04:04:13.724736 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724735    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724759    6751 cpu_manager.go:414] "containerMap: removing container" podUID="e1069831-769d-4c65-b634-f6c12d2e00b9" containerName="main"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724767    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1069831-769d-4c65-b634-f6c12d2e00b9" containerName="main"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724777    6751 cpu_manager.go:414] "containerMap: removing container" podUID="a174f8b4-9d90-407f-8914-107ddd97d3db" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724783    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a174f8b4-9d90-407f-8914-107ddd97d3db" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724799    6751 cpu_manager.go:414] "containerMap: removing container" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724806    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724817    6751 cpu_manager.go:414] "containerMap: removing container" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724824    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724839    6751 cpu_manager.go:414] "containerMap: removing container" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724846    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724862    6751 cpu_manager.go:414] "containerMap: removing container" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724871    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724889    6751 cpu_manager.go:414] "containerMap: removing container" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724894    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724902    6751 cpu_manager.go:414] "containerMap: removing container" podUID="ef77f1d6-3638-4e63-ad6c-b0e8ea320df2" containerName="revoke-keys"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724907    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef77f1d6-3638-4e63-ad6c-b0e8ea320df2" containerName="revoke-keys"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724918    6751 cpu_manager.go:414] "containerMap: removing container" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724926    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724935    6751 cpu_manager.go:414] "containerMap: removing container" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy"
Nov 04 04:04:13.724943 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.724940    6751 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725024    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef77f1d6-3638-4e63-ad6c-b0e8ea320df2" containerName="revoke-keys"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725033    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="74ceb3c9-eee7-4329-ae34-2dc9191b1908" containerName="istio-proxy"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725040    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725045    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="4697d051-eb18-4ed0-92c9-408a109a5bf1" containerName="istio-proxy"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725051    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="5931acd2-4ef3-4a72-b76f-c792ffee8c4a" containerName="istio-proxy"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725058    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="56656b54-8f49-4a46-92a6-59400b873509" containerName="istio-proxy"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725067    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1069831-769d-4c65-b634-f6c12d2e00b9" containerName="main"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725074    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="a174f8b4-9d90-407f-8914-107ddd97d3db" containerName="istio-proxy"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725082    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="52a8de46-9185-43a4-80c5-cfcedf3dd676" containerName="istio-proxy"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725090    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy"
Nov 04 04:04:13.725613 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.725153    6751 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8e1215b-ab44-4926-81e7-dcb61def4ad5" containerName="istio-proxy"
Nov 04 04:04:13.728130 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.728116    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.734115 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.733954    6751 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-26w7b\"/\"default-dockercfg-4hjnm\""
Nov 04 04:04:13.735838 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.735817    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-26w7b\"/\"kube-root-ca.crt\""
Nov 04 04:04:13.735946 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.735858    6751 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-26w7b\"/\"openshift-service-ca.crt\""
Nov 04 04:04:13.737906 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.737886    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"]
Nov 04 04:04:13.754012 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.753983    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-sys\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.754116 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.754044    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-podres\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.754169 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.754133    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-proc\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.754219 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.754189    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-lib-modules\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.754295 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.754277    6751 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z468b\" (UniqueName: \"kubernetes.io/projected/3e053ce4-5856-4363-9722-5aadb86abe1c-kube-api-access-z468b\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.855317 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.855290    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-sys\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.855441 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.855335    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-podres\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.855441 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.855384    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-proc\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.855441 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.855405    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-sys\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.855595 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.855465    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-lib-modules\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.855595 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.855472    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-podres\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.855595 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.855483    6751 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z468b\" (UniqueName: \"kubernetes.io/projected/3e053ce4-5856-4363-9722-5aadb86abe1c-kube-api-access-z468b\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.855595 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.855471    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-proc\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.855595 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.855526    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e053ce4-5856-4363-9722-5aadb86abe1c-lib-modules\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:13.867912 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:13.867889    6751 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z468b\" (UniqueName: \"kubernetes.io/projected/3e053ce4-5856-4363-9722-5aadb86abe1c-kube-api-access-z468b\") pod \"perf-node-gather-daemonset-8jcxq\" (UID: \"3e053ce4-5856-4363-9722-5aadb86abe1c\") " pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:14.042400 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:14.042378    6751 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:14.316193 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:14.316169    6751 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"]
Nov 04 04:04:14.318054 ip-10-0-129-253 kubenswrapper[6751]: W1104 04:04:14.318029    6751 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3e053ce4_5856_4363_9722_5aadb86abe1c.slice/crio-4fef0998fbfdd54a422695d9da1f03f6ec461bd887085b56825d54305a2ec4c3 WatchSource:0}: Error finding container 4fef0998fbfdd54a422695d9da1f03f6ec461bd887085b56825d54305a2ec4c3: Status 404 returned error can't find the container with id 4fef0998fbfdd54a422695d9da1f03f6ec461bd887085b56825d54305a2ec4c3
Nov 04 04:04:14.547413 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:14.547375    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq" event={"ID":"3e053ce4-5856-4363-9722-5aadb86abe1c","Type":"ContainerStarted","Data":"8b50ef6acdb1807997ca6d30b3c1010e9a354757b585f212fb506f5ce596945b"}
Nov 04 04:04:14.548107 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:14.547432    6751 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq" event={"ID":"3e053ce4-5856-4363-9722-5aadb86abe1c","Type":"ContainerStarted","Data":"4fef0998fbfdd54a422695d9da1f03f6ec461bd887085b56825d54305a2ec4c3"}
Nov 04 04:04:14.548107 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:14.547543    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:14.568402 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:14.568318    6751 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq" podStartSLOduration=1.568305671 podStartE2EDuration="1.568305671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 04:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 04:04:14.566660828 +0000 UTC m=+848.900598723" watchObservedRunningTime="2026-11-04 04:04:14.568305671 +0000 UTC m=+848.902243549"
Nov 04 04:04:16.058744 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:16.058719    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-795f6cf888-8798b_2fc79c0c-7d55-46f7-a19e-09f92b6c8f8a/console/0.log"
Nov 04 04:04:16.093838 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:16.093811    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-mv47f_83f09eac-bda3-430a-958a-2a9bef3ad8e0/download-server/0.log"
Nov 04 04:04:18.018531 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:18.018502    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lcn4v_4f25400e-aea9-43e3-b17f-0a68a76a4632/dns/0.log"
Nov 04 04:04:18.047420 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:18.047383    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lcn4v_4f25400e-aea9-43e3-b17f-0a68a76a4632/kube-rbac-proxy/0.log"
Nov 04 04:04:18.194636 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:18.194606    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5n7tp_4cc23bbb-cc49-4e50-a94a-df074fcd4d3e/dns-node-resolver/0.log"
Nov 04 04:04:18.907258 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:18.907229    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7jmdr_d4639182-9c75-4c83-81aa-e8e59d52fe6e/node-ca/0.log"
Nov 04 04:04:20.524545 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:20.524514    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-82hvc_b030923a-85f1-432c-accd-ac50ab54f8ee/discovery/0.log"
Nov 04 04:04:20.551102 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:20.551079    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bd469f65f-4x7zx_fc13f463-fe9a-4482-9d0a-fac5a2b845bb/kube-auth-proxy/0.log"
Nov 04 04:04:20.563363 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:20.563335    6751 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-26w7b/perf-node-gather-daemonset-8jcxq"
Nov 04 04:04:20.650369 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:20.650343    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-96lb9_13ffed84-554f-440a-8016-20e8344e5135/istio-proxy/0.log"
Nov 04 04:04:20.795036 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:20.794961    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-wjbj2_7427f8fe-4e90-426b-9e23-7054641123b3/istio-proxy/0.log"
Nov 04 04:04:20.830460 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:20.830435    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-xrqpp_90da3418-09f9-4d87-b646-75914ed6a3bc/istio-proxy/0.log"
Nov 04 04:04:21.086977 ip-10-0-129-253 kubenswrapper[6751]: E1104 04:04:21.086897    6751 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qkmp2" containerName="extractor"
Nov 04 04:04:22.091531 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:22.091497    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s4h9x_68df7a49-2cdd-46c3-8730-9447708be784/serve-healthcheck-canary/0.log"
Nov 04 04:04:22.810687 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:22.810663    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qkmp2_9062ed54-5f71-4f96-8274-20ab5a38a9a0/kube-rbac-proxy/0.log"
Nov 04 04:04:22.837825 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:22.837796    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qkmp2_9062ed54-5f71-4f96-8274-20ab5a38a9a0/exporter/0.log"
Nov 04 04:04:22.864576 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:22.864534    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qkmp2_9062ed54-5f71-4f96-8274-20ab5a38a9a0/extractor/0.log"
Nov 04 04:04:25.391051 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:25.391022    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-82hvc_b030923a-85f1-432c-accd-ac50ab54f8ee/discovery/0.log"
Nov 04 04:04:25.403712 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:25.403681    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bd469f65f-4x7zx_fc13f463-fe9a-4482-9d0a-fac5a2b845bb/kube-auth-proxy/0.log"
Nov 04 04:04:25.466417 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:25.466379    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-96lb9_13ffed84-554f-440a-8016-20e8344e5135/istio-proxy/0.log"
Nov 04 04:04:25.561956 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:25.561934    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-wjbj2_7427f8fe-4e90-426b-9e23-7054641123b3/istio-proxy/0.log"
Nov 04 04:04:25.582483 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:25.582451    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-xrqpp_90da3418-09f9-4d87-b646-75914ed6a3bc/istio-proxy/0.log"
Nov 04 04:04:25.891246 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:25.891219    6751 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49hmt_d6bb8b1d-e87b-4167-a84a-92e9815f2206/konnectivity-agent/0.log" limit=1024
Nov 04 04:04:26.090536 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:26.090514    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-74f9c8978b-mqnj9_49f01ef9-ae7f-4fbc-85ed-72c857421e68/manager/0.log"
Nov 04 04:04:26.118087 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:26.118061    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-74f9c8978b-mqnj9_49f01ef9-ae7f-4fbc-85ed-72c857421e68/copy-manifests/0.log"
Nov 04 04:04:27.383571 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:27.383546    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-5f85d5d778-bz64z_969c5e32-5489-4455-8846-57e78155faf5/manager/0.log"
Nov 04 04:04:27.651955 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:27.651878    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-82hvc_b030923a-85f1-432c-accd-ac50ab54f8ee/discovery/0.log"
Nov 04 04:04:27.675797 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:27.675774    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bd469f65f-4x7zx_fc13f463-fe9a-4482-9d0a-fac5a2b845bb/kube-auth-proxy/0.log"
Nov 04 04:04:27.734803 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:27.734781    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-96lb9_13ffed84-554f-440a-8016-20e8344e5135/istio-proxy/0.log"
Nov 04 04:04:27.826615 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:27.826568    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-wjbj2_7427f8fe-4e90-426b-9e23-7054641123b3/istio-proxy/0.log"
Nov 04 04:04:27.848668 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:27.848648    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d64fcddb7-xrqpp_90da3418-09f9-4d87-b646-75914ed6a3bc/istio-proxy/0.log"
Nov 04 04:04:28.059079 ip-10-0-129-253 kubenswrapper[6751]: I1104 04:04:28.058995    6751 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-79b649855b-9p994_1636e316-ede4-4230-a002-46b1fcea8469/manager/0.log"