Nov 04 01:52:54.745668 ip-10-0-142-191 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 04 01:52:54.745680 ip-10-0-142-191 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 04 01:52:54.745687 ip-10-0-142-191 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 04 01:52:54.745946 ip-10-0-142-191 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 04 01:53:04.780346 ip-10-0-142-191 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 04 01:53:04.780365 ip-10-0-142-191 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot bd868ee26be445edabf1dbd4a6c45073 --
Nov 04 01:54:56.760790 ip-10-0-142-191 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 01:54:57.153522 ip-10-0-142-191 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:54:57.153522 ip-10-0-142-191 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 01:54:57.153522 ip-10-0-142-191 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:54:57.153522 ip-10-0-142-191 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:54:57.159524 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159424    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:57.159524 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159520    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:57.159524 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159524    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:57.159524 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159527    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:57.159524 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159530    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159533    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159536    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159538    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159541    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159543    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159546    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159549    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159552    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159555    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159557    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159560    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159563    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159566    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159568    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159571    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159573    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159579    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159582    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159586    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:57.159663 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159591    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159594    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159597    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159600    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159603    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159606    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159609    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159611    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159614    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159617    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159619    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159622    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159624    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159627    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159630    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159633    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159635    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159638    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159641    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159644    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:57.160168 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159646    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159649    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159651    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159654    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159656    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159659    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159662    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159666    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159668    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159671    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159673    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159676    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159679    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159682    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159684    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159687    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159689    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159692    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159695    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:57.160658 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159699    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159703    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159706    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159708    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159711    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159713    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159716    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159720    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159722    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159731    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159734    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159736    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159739    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159741    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159744    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159746    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159748    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159751    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159753    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159756    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:57.161130 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159758    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159761    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159763    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159765    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159768    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159770    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159773    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159775    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159778    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159780    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159783    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159785    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159787    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159790    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159792    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159796    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159798    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159801    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159803    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:57.161616 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159806    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159809    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159811    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.159814    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160620    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160628    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160631    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160634    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160638    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160641    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160643    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160648    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160652    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160655    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160658    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160661    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160663    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160666    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160670    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:57.162110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160673    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160675    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160678    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160680    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160683    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160685    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160688    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160691    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160693    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160696    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160699    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160701    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160704    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160707    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160710    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160712    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160715    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160718    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160721    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:57.162592 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160723    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160725    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160728    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160731    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160733    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160735    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160738    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160740    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160743    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160746    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160748    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160751    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160754    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160756    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160759    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160761    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160763    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160766    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160769    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160771    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:57.163036 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160774    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160777    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160779    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160782    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160785    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160787    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160790    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160793    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160795    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160798    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160800    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160803    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160806    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160808    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160810    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160813    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160815    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160818    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160820    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160823    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:57.163518 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160825    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160827    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160830    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160832    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160834    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160837    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160839    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160841    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160844    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160848    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160851    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160854    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160856    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160859    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160862    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160865    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160868    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160870    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160873    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:57.164024 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160876    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160878    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160880    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160883    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160886    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160890    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160892    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160895    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160897    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160899    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160902    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160904    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.160907    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161427    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161433    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161437    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161441    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161443    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161446    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:57.164478 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161449    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161451    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161455    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161457    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161460    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161463    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161466    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161469    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161472    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161475    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161477    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161480    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161483    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161485    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161488    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161490    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161493    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161495    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161498    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161501    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161520    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:57.164946 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161523    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161526    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161528    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161531    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161533    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161536    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161538    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161541    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161544    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161546    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161549    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161551    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161554    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161556    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161559    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161561    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161564    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161567    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161571    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:57.165452 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161573    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161576    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161578    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161581    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161583    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161586    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161588    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161590    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161593    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161595    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161598    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161601    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161603    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161606    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161609    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161611    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161614    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161616    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161619    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161621    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:57.165915 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161624    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161627    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161630    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161633    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161635    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161638    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161640    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161642    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161646    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161650    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161655    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161658    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161661    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161663    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161666    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161668    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161671    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161673    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161676    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:57.166431 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161678    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161682    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161685    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161688    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161691    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161702    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161705    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161707    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161710    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161712    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161715    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161717    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161719    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161722    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161726    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161728    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161731    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161733    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161736    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:57.166900 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161738    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.161740    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162734    2581 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162744    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162753    2581 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162758    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162762    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162766    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162770    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162774    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162778    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162781    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162784    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162787    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162790    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162793    2581 flags.go:64] FLAG: --cgroup-root=""
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162796    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162799    2581 flags.go:64] FLAG: --client-ca-file=""
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162802    2581 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162806    2581 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162810    2581 flags.go:64] FLAG: --cluster-domain=""
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162813    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162816    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162818    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 01:54:57.167347 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162822    2581 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162826    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162828    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162831    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162834    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162837    2581 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162840    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162843    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162846    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162849    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162853    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162856    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162859    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162865    2581 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162868    2581 flags.go:64] FLAG: --enable-server="true"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162871    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162875    2581 flags.go:64] FLAG: --event-burst="100"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162878    2581 flags.go:64] FLAG: --event-qps="50"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162881    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162885    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162888    2581 flags.go:64] FLAG: --eviction-hard=""
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162892    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162895    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162898    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162901    2581 flags.go:64] FLAG: --eviction-soft=""
Nov 04 01:54:57.167931 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162904    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162907    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162909    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162913    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162915    2581 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162918    2581 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162921    2581 flags.go:64] FLAG: --feature-gates=""
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162930    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162933    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162937    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162940    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162943    2581 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162946    2581 flags.go:64] FLAG: --help="false"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162948    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162951    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162954    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162957    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162960    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162963    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162966    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162970    2581 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162973    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162976    2581 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 01:54:57.168587 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162978    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162981    2581 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162984    2581 flags.go:64] FLAG: --kube-reserved=""
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162987    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162990    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162993    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162995    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.162998    2581 flags.go:64] FLAG: --lock-file=""
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163001    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163004    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163007    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163013    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163016    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163019    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163022    2581 flags.go:64] FLAG: --logging-format="text"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163025    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163028    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163031    2581 flags.go:64] FLAG: --manifest-url=""
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163034    2581 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163038    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163042    2581 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163046    2581 flags.go:64] FLAG: --max-pods="110"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163049    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163052    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163055    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 01:54:57.169136 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163058    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163061    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163064    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163066    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163075    2581 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163078    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163081    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163084    2581 flags.go:64] FLAG: --pod-cidr=""
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163087    2581 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163090    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163093    2581 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163096    2581 flags.go:64] FLAG: --port="10250"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163099    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163101    2581 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0cc1e9cb2e0ca73a7"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163104    2581 flags.go:64] FLAG: --qos-reserved=""
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163107    2581 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163110    2581 flags.go:64] FLAG: --register-node="true"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163113    2581 flags.go:64] FLAG: --register-with-taints=""
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163117    2581 flags.go:64] FLAG: --registry-burst="10"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163119    2581 flags.go:64] FLAG: --registry-qps="5"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163122    2581 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163125    2581 flags.go:64] FLAG: --reserved-memory=""
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163130    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163133    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163136    2581 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163138    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 01:54:57.169765 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163141    2581 flags.go:64] FLAG: --runonce="false"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163144    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163147    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163149    2581 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163152    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163155    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163158    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163161    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163167    2581 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163170    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163173    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163176    2581 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163179    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163182    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163185    2581 flags.go:64] FLAG: --system-cgroups=""
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163188    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163194    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163197    2581 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163200    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163204    2581 flags.go:64] FLAG: --tls-min-version=""
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163207    2581 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163210    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163213    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163216    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163218    2581 flags.go:64] FLAG: --v="2"
Nov 04 01:54:57.170385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163223    2581 flags.go:64] FLAG: --version="false"
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163226    2581 flags.go:64] FLAG: --vmodule=""
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163230    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163234    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163371    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163374    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163377    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163380    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163383    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163385    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163388    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163390    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163392    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163395    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163397    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163402    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163405    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163411    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163414    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163416    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163419    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:57.170983 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163421    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163424    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163426    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163429    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163432    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163434    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163436    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163439    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163442    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163444    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163447    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163449    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163452    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163456    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163459    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163463    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163465    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163468    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163471    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:57.171481 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163473    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163475    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163478    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163480    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163482    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163485    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163487    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163492    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163496    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163501    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163517    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163520    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163523    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163526    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163529    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163532    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163534    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163537    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163539    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:57.172067 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163542    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163544    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163547    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163549    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163551    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163554    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163556    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163559    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163561    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163564    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163567    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163569    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163572    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163574    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163577    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163579    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163582    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163585    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163587    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163589    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:57.172813 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163593    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163595    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163599    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163601    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163604    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163606    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163609    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163611    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163614    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163616    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163619    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163621    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163624    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163626    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163628    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163631    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163633    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163636    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163638    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:57.173329 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163641    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163643    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163646    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163648    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163651    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163653    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163656    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163658    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163661    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163663    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163666    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163668    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.163694    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163730    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163736    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163739    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:57.173808 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163741    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163744    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163747    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163749    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163752    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163754    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163757    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163759    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163762    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163765    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163767    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163770    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163772    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163775    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163777    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163779    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163782    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163784    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163787    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163789    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:57.174188 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163792    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163803    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163805    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163808    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163811    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163813    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163816    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163818    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163821    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163823    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163827    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163830    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163833    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163835    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163838    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163840    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163843    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163845    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163848    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:57.174682 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163850    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163853    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163856    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163858    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163860    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163863    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163865    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163868    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163870    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163873    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163875    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163878    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163881    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163883    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163886    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163889    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163891    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163894    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163897    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163899    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:57.175134 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163901    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163904    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163906    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163910    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163913    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163916    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163918    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163922    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163925    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163927    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163930    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163932    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163935    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163938    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163941    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163944    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163948    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163951    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163954    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:57.175637 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163957    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163959    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163962    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163964    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163967    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163969    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163971    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163974    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163976    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163981    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163984    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163988    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163991    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163995    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.163998    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.164000    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.164004    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.164007    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.164010    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:57.176085 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.164013    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.164016    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.164019    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.164021    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.164023    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.164026    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.171276    2581 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.171293    2581 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.171308    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.171313    2581 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171371    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171375    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171379    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171382    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171385    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171388    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171391    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171393    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171396    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:57.176693 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171398    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171401    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171404    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171406    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171409    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171411    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171414    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171416    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171419    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171421    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171424    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171426    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171428    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171431    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171433    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171436    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171438    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171441    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171443    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:57.177140 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171446    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171451    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171454    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171456    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171458    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171461    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171463    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171466    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171468    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171471    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171473    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171476    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171478    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171481    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171483    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171486    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171489    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171492    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171497    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:57.177593 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171501    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171523    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171528    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171532    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171536    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171539    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171541    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171544    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171546    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171549    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171551    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171554    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171556    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171559    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171561    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171565    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171567    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171570    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171572    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171575    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:57.178038 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171577    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171579    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171582    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171584    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171587    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171589    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171592    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171594    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171596    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171599    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171601    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171604    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171606    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171609    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171611    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171614    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171616    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171619    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171621    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171624    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:57.178551 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171626    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171628    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171631    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171635    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171639    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171642    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171645    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171648    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171652    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171655    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171657    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171660    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171663    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171665    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171667    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171670    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171672    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171675    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:57.179035 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.171677    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.171706    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174012    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174024    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174028    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174030    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174033    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174036    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174039    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174043    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174045    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174049    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174051    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174054    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174057    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174059    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:57.179443 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174062    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174064    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174067    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174070    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174072    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174075    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174077    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174080    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174082    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174085    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174087    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174089    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174092    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174095    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174097    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174100    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174102    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174105    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174109    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:57.179923 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174112    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174115    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174118    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174120    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174123    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174127    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174132    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174135    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174139    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174141    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174146    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174148    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174152    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174154    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174158    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174161    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174164    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174168    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174170    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174174    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:57.180372 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174178    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174181    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174184    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174186    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174189    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174192    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174195    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174197    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174200    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174202    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174205    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174209    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174211    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174214    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174216    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174219    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174222    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174224    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174227    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:57.180846 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174230    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174232    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174235    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174237    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174240    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174242    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174245    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174247    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174250    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174252    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174254    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174258    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174260    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174263    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174265    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174268    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174271    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174275    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174278    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174281    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:57.181288 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174284    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174287    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174290    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174292    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174296    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174298    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174301    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174303    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174306    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174309    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174311    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174314    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174316    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174319    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.174348    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174411    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:54:57.181769 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174416    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174419    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174424    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174428    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174432    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174434    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174437    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174440    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174443    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174445    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174448    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174451    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174453    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174455    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174458    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174460    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174464    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174466    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174468    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:54:57.182147 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174471    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174473    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174477    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174479    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174482    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174484    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174487    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174489    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174492    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174494    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174496    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174499    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174516    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174520    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174523    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174525    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174528    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174530    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174533    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174535    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:54:57.182625 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174540    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174544    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174547    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174550    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174560    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174563    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174566    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174569    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174572    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174574    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174576    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174579    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174582    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174585    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174587    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174590    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174593    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174595    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174598    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174600    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:54:57.183109 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174603    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174605    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174608    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174610    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174613    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174615    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174618    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174620    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174622    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174625    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174627    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174630    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174632    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174634    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174637    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174640    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174643    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174645    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174648    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:54:57.183674 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174650    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174652    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174655    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174657    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174659    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174662    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174664    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174667    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174669    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174672    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174674    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174677    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174679    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174682    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174684    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174688    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174691    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174694    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174696    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174699    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:54:57.184110 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174701    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174704    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174706    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174709    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174711    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174713    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:57.174716    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.174769    2581 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.177119    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.178084    2581 server.go:1014] "Starting client certificate rotation"
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.178178    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.178212    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.180278    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:54:57.184583 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.181447    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:54:57.191151 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.191137    2581 log.go:25] "Validated CRI v1 runtime API"
Nov 04 01:54:57.195073 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.195058    2581 log.go:25] "Validated CRI v1 image API"
Nov 04 01:54:57.195980 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.195945    2581 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 01:54:57.199276 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.199255    2581 fs.go:135] Filesystem UUIDs: map[4de28615-663a-4331-b88e-e7c5b8c95908:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 fd248c9a-c51a-405e-963f-90e25cf0a42c:/dev/nvme0n1p3]
Nov 04 01:54:57.199345 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.199278    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 01:54:57.205679 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.205560    2581 manager.go:217] Machine: {Timestamp:2026-11-04 01:54:57.2041714 +0000 UTC m=+0.335861175 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3080938 MemoryCapacity:32812023808 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec254f6ad69576a68578c0ab18e419a3 SystemUUID:ec254f6a-d695-76a6-8578-c0ab18e419a3 BootID:bd868ee2-6be4-45ed-abf1-dbd4a6c45073 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406011904 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:78:0a:f5:9f:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:78:0a:f5:9f:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:aa:02:66:5b:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812023808 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 01:54:57.205679 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.205677    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 01:54:57.205820 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.205752    2581 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 01:54:57.209313 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.209294    2581 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 01:54:57.209501 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.209315    2581 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-191.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 01:54:57.209557 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.209520    2581 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 01:54:57.209557 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.209529    2581 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 01:54:57.209617 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.209566    2581 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:54:57.209617 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.209583    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:54:57.210814 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.210803    2581 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 01:54:57.215192 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.215179    2581 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 01:54:57.215315 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.215306    2581 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 01:54:57.216227 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.216216    2581 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 01:54:57.216267 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.216236    2581 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 01:54:57.216267 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.216247    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 01:54:57.216267 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.216256    2581 kubelet.go:405] "Adding apiserver pod source"
Nov 04 01:54:57.216267 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.216264    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 01:54:57.217225 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.217201    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:54:57.224326 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.224306    2581 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 01:54:57.231412 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.231398    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:54:57.232327 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.232314    2581 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 01:54:57.232366 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.232345    2581 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 01:54:57.232496 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.232486    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 01:54:57.232548 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.232517    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 01:54:57.232548 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.232527    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 01:54:57.232548 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.232533    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 01:54:57.232548 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.232538    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 01:54:57.232548 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.232544    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 01:54:57.233281 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.233272    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 01:54:57.233311 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.233281    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 01:54:57.233311 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.233293    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 01:54:57.233311 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.233299    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 01:54:57.233311 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.233307    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 01:54:57.233496 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.233315    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 01:54:57.234950 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.234940    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 01:54:57.234950 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.234950    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 01:54:57.238983 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.238959    2581 server.go:1263] "Started kubelet"
Nov 04 01:54:57.239099 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.239009    2581 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 01:54:57.239153 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.239083    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 01:54:57.239153 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.239144    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 01:54:57.239802 ip-10-0-142-191 systemd[1]: Started Kubernetes Kubelet.
Nov 04 01:54:57.240663 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.240634    2581 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 01:54:57.241903 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.241879    2581 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 01:54:57.244545 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.244488    2581 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 01:54:57.248077 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.248060    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 01:54:57.248673 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.248654    2581 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 01:54:57.248853 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.248806    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:57.249385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249368    2581 factory.go:55] Registering systemd factory
Nov 04 01:54:57.249385 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249385    2581 factory.go:223] Registration of the systemd container factory successfully
Nov 04 01:54:57.249529 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249451    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 01:54:57.249529 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249452    2581 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 01:54:57.249529 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249468    2581 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 01:54:57.249684 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249580    2581 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 01:54:57.249684 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249590    2581 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 01:54:57.249786 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249738    2581 factory.go:153] Registering CRI-O factory
Nov 04 01:54:57.249786 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249754    2581 factory.go:223] Registration of the crio container factory successfully
Nov 04 01:54:57.249881 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249823    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 01:54:57.249881 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249842    2581 factory.go:103] Registering Raw factory
Nov 04 01:54:57.249881 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.249856    2581 manager.go:1199] Started watching for new ooms in manager
Nov 04 01:54:57.251080 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.251066    2581 manager.go:319] Starting recovery of all containers
Nov 04 01:54:57.264595 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.264531    2581 manager.go:324] Recovery completed
Nov 04 01:54:57.269118 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.269106    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:57.270741 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.270722    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:57.270811 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.270757    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:57.270811 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.270772    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:57.271263 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.271249    2581 cpu_manager.go:226] "Starting" policy="none"
Nov 04 01:54:57.271263 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.271261    2581 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 01:54:57.271373 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.271280    2581 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 01:54:57.271373 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.271299    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:57.272029 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.272014    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:57.272074 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.272041    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:57.272074 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.272051    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:57.273586 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.273571    2581 policy_none.go:50] "Start"
Nov 04 01:54:57.273649 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.273589    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 01:54:57.273649 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.273599    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 01:54:57.275589 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.275578    2581 policy_none.go:44] "Start"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.311126    2581 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.316206    2581 manager.go:341] "Starting Device Plugin manager"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.316230    2581 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.316239    2581 server.go:87] "Starting device plugin registration server"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.316435    2581 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.316444    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.316589    2581 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.316685    2581 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.316695    2581 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.317181    2581 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.317230    2581 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.323714    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.325048    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.325077    2581 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.325095    2581 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 01:54:57.331782 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.325161    2581 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 01:54:57.382777 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.382632    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-191.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 01:54:57.417339 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.417280    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:57.418190 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.418173    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:57.418285 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.418204    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:57.418285 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.418217    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:57.418285 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.418240    2581 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.425742 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.425720    2581 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"]
Nov 04 01:54:57.425802 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.425778    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:57.426445 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.426429    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:57.426496 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.426464    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:57.426496 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.426474    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:57.428615 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.428602    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:57.428767 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.428751    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.428831 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.428789    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:57.429297 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.429281    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:57.429377 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.429312    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:57.429377 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.429322    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:57.429377 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.429344    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:57.429377 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.429368    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:57.429600 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.429383    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:57.431407 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.431391    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.431466 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.431425    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:57.431948 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.431826    2581 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-191.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 01:54:57.432767 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.431882    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-191.us-west-1.compute.internal.18e4b4c2132e82fc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-191.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-191.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:54:57.238770428 +0000 UTC m=+0.370460202,LastTimestamp:2026-11-04 01:54:57.238770428 +0000 UTC m=+0.370460202,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-191.us-west-1.compute.internal,}"
Nov 04 01:54:57.433880 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.433862    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:57.433966 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.433895    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:57.433966 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.433905    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:57.450826 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.450802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.450903 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.450838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.450903 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.450888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f88ddff8fbb738ea2fba74409a8de90-config\") pod \"kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"9f88ddff8fbb738ea2fba74409a8de90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.455379 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.455359    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 01:54:57.455436 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.455413    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sc5wz"
Nov 04 01:54:57.461155 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.461139    2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.463024 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.463009    2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.487151 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.487099    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Nov 04 01:54:57.487151 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.487111    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-191.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 04 01:54:57.487285 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.487149    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 04 01:54:57.487285 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.487114    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-191.us-west-1.compute.internal.18e4b4c2151652b1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-191.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-191.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-191.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:54:57.270739633 +0000 UTC m=+0.402429417,LastTimestamp:2026-11-04 01:54:57.270739633 +0000 UTC m=+0.402429417,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-191.us-west-1.compute.internal,}"
Nov 04 01:54:57.487369 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.487203    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 04 01:54:57.551579 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.551556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.551662 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.551583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.551662 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.551621    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f88ddff8fbb738ea2fba74409a8de90-config\") pod \"kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"9f88ddff8fbb738ea2fba74409a8de90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.551662 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.551654    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.551755 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.551660    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.551755 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.551657    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f88ddff8fbb738ea2fba74409a8de90-config\") pod \"kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"9f88ddff8fbb738ea2fba74409a8de90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.578378 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.578352    2581 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-191.us-west-1.compute.internal\" not found" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.735639 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.735564    2581 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-191.us-west-1.compute.internal\" not found" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.766141 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.766121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.767245 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.767230    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.779176 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.779163    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:57.780037 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.780022    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:57.780107 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.780053    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:57.780107 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.780064    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:57.780107 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.780089    2581 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.837118 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:57.837094    2581 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:57.837118 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.837120    2581 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-191.us-west-1.compute.internal\": node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:57.957247 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:57.957217    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:58.057994 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.057916    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:58.158269 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.158247    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:58.178423 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.178391    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 01:54:58.178651 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.178568    2581 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://ab045e258af594899a140baaafc158e1-d72efd8280e6925c.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-142-191.us-west-1.compute.internal.18e4b4c2151652b1\": read tcp 10.0.142.191:42802->3.143.52.118:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-142-191.us-west-1.compute.internal.18e4b4c2151652b1  default   5095 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-191.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-191.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-191.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:54:57 +0000 UTC,LastTimestamp:2026-11-04 01:54:57.42644621 +0000 UTC m=+0.558135984,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-191.us-west-1.compute.internal,}"
Nov 04 01:54:58.259336 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.259307    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:58.359907 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.359837    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:58.460801 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.460766    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:58.460801 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.460784    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 01:49:58 +0000 UTC" deadline="2028-06-13 03:09:50.163980363 +0000 UTC"
Nov 04 01:54:58.460961 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.460816    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14089h14m51.703167742s"
Nov 04 01:54:58.561380 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.561350    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:58.570189 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.570168    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:54:58.571090 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.571074    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:54:58.571192 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.571109    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:54:58.571192 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.571123    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:54:58.647479 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.647407    2581 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:54:58.661921 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.661897    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:58.755409 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.755382    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:54:58.762335 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.762310    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:58.863105 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.862730    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:58.874943 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:58.874915    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23463ee86a57f13d80b429c8e67d6e17.slice/crio-b3eb0ec95449a0a40f3c1fb1812e3cec836189a48d2c75da55cf0a38c27799ee WatchSource:0}: Error finding container b3eb0ec95449a0a40f3c1fb1812e3cec836189a48d2c75da55cf0a38c27799ee: Status 404 returned error can't find the container with id b3eb0ec95449a0a40f3c1fb1812e3cec836189a48d2c75da55cf0a38c27799ee
Nov 04 01:54:58.875171 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:58.875148    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f88ddff8fbb738ea2fba74409a8de90.slice/crio-a700b7f4b57af9ece70802725c34dec353f916ec71c513e5d3d68d54c4ed7f3e WatchSource:0}: Error finding container a700b7f4b57af9ece70802725c34dec353f916ec71c513e5d3d68d54c4ed7f3e: Status 404 returned error can't find the container with id a700b7f4b57af9ece70802725c34dec353f916ec71c513e5d3d68d54c4ed7f3e
Nov 04 01:54:58.880731 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.880714    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:54:58.888172 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:58.888155    2581 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:54:58.963623 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:58.963544    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:54:59.036273 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.036245    2581 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:54:59.049333 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.049316    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:59.150200 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.150166    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:54:59.151262 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.151249    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:54:59.209335 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.209312    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:54:59.217462 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.217404    2581 apiserver.go:52] "Watching apiserver"
Nov 04 01:54:59.249670 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.249646    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 01:54:59.323469 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.323446    2581 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 01:54:59.323799 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.323781    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-4k7bb","openshift-dns/node-resolver-tm85n","openshift-image-registry/node-ca-fgzvx","openshift-multus/multus-mgw25","openshift-network-diagnostics/network-check-target-jnspf","kube-system/konnectivity-agent-5v5x7","kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm","openshift-multus/network-metrics-daemon-6g7qh","openshift-network-operator/iptables-alerter-rx5p4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal","openshift-multus/multus-additional-cni-plugins-c76wf","openshift-ovn-kubernetes/ovnkube-node-6gccn","kube-system/global-pull-secret-syncer-dlxgx"]
Nov 04 01:54:59.327797 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.327777    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.330194 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.330172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.332453 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.332436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.336116 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.336091    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.338363 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.338344    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:54:59.338465 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.338421    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:54:59.340733 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.340717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:54:59.343108 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.343089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:54:59.343189 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.343166    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:54:59.345534 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.345518    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:54:59.345611 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.345579    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:54:59.347859 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.347835    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.350266 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.350242    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.352713 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.352691    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.355120 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.355103    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.357959 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.357912    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"9f88ddff8fbb738ea2fba74409a8de90","Type":"ContainerStarted","Data":"a700b7f4b57af9ece70802725c34dec353f916ec71c513e5d3d68d54c4ed7f3e"}
Nov 04 01:54:59.358053 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.357972    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"23463ee86a57f13d80b429c8e67d6e17","Type":"ContainerStarted","Data":"b3eb0ec95449a0a40f3c1fb1812e3cec836189a48d2c75da55cf0a38c27799ee"}
Nov 04 01:54:59.362887 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.362867    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:54:59.362969 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.362894    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8g55\" (UniqueName: \"kubernetes.io/projected/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-kube-api-access-g8g55\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.362969 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.362911    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-log-socket\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.362969 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.362927    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87120866-99b9-46d6-9883-6244b38c38ae-ovn-node-metrics-cert\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.362969 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.362943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cni-binary-copy\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.363178 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363010    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-hostroot\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.363178 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8k7f\" (UniqueName: \"kubernetes.io/projected/30ad9387-03e3-49d0-8ff8-b4959c6f343f-kube-api-access-m8k7f\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:54:59.363178 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-conf\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.363178 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-lib-modules\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.363399 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363205    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-tmp\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.363399 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b94155ab-70f0-4efa-a01c-7303db0d8cb5-serviceca\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.363399 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-systemd\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.363399 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363295    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-ovn\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.363399 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-socket-dir-parent\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.363399 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363351    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-multus\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.363399 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363419    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcwq8\" (UniqueName: \"kubernetes.io/projected/b94155ab-70f0-4efa-a01c-7303db0d8cb5-kube-api-access-zcwq8\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-tmp-dir\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363478    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-run\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kchqw\" (UniqueName: \"kubernetes.io/projected/87120866-99b9-46d6-9883-6244b38c38ae-kube-api-access-kchqw\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363560    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-k8s-cni-cncf-io\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-kubelet\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-kubernetes\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-systemd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363660    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-multus-certs\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-kubelet-config\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b94155ab-70f0-4efa-a01c-7303db0d8cb5-host\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-etc-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.363780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363772    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-env-overrides\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-dbus\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-netns\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px9wr\" (UniqueName: \"kubernetes.io/projected/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-kube-api-access-px9wr\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363857    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-var-lib-kubelet\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363878    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcrk5\" (UniqueName: \"kubernetes.io/projected/1ca946c4-34f5-4443-9ecf-e634055dacc0-kube-api-access-mcrk5\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-netd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-bin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363959    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-hosts-file\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.363985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-host\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-slash\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364022    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-var-lib-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-node-log\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cnibin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-tuned\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.364439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364183    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-daemon-config\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364216    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-iptables-alerter-script\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-script-lib\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364273    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfca312e-b151-4c3a-99bd-dd820925264b-agent-certs\") pod \"konnectivity-agent-5v5x7\" (UID: \"dfca312e-b151-4c3a-99bd-dd820925264b\") " pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364307    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-bin\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364325    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-netns\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-etc-kubernetes\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfca312e-b151-4c3a-99bd-dd820925264b-konnectivity-ca\") pod \"konnectivity-agent-5v5x7\" (UID: \"dfca312e-b151-4c3a-99bd-dd820925264b\") " pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364442    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-modprobe-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364529    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-config\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-conf-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xjbg\" (UniqueName: \"kubernetes.io/projected/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-kube-api-access-7xjbg\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-sys\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.365322 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-system-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.366217 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-host-slash\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.366217 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364863    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysconfig\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.366217 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-kubelet\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.366217 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.364914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-systemd-units\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.389277 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.389250    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 01:54:59.389478 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.389458    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nrwjt\""
Nov 04 01:54:59.389583 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.389561    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r7nnn\""
Nov 04 01:54:59.390161 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.390136    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 01:54:59.390254 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.390224    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 01:54:59.390254 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.390225    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:54:59.390254 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.390243    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 01:54:59.390421 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.390282    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 01:54:59.390421 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.390359    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 01:54:59.390527 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.390418    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 01:54:59.390527 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.390469    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 01:54:59.390527 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.390500    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 01:54:59.390527 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.390507    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 01:54:59.393749 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.393730    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 01:54:59.452388 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.452356    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 01:54:59.465202 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465172    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-iptables-alerter-script\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.465281 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-script-lib\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.465281 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465229    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.465281 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-binary-copy\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.465418 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfca312e-b151-4c3a-99bd-dd820925264b-agent-certs\") pod \"konnectivity-agent-5v5x7\" (UID: \"dfca312e-b151-4c3a-99bd-dd820925264b\") " pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:54:59.465474 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-bin\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.465527 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-netns\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.465527 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-etc-kubernetes\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.465659 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-system-cni-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.465659 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465532    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-bin\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.465659 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465595    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.465659 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465591    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-etc-kubernetes\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.465659 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465574    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-netns\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.465659 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfca312e-b151-4c3a-99bd-dd820925264b-konnectivity-ca\") pod \"konnectivity-agent-5v5x7\" (UID: \"dfca312e-b151-4c3a-99bd-dd820925264b\") " pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-modprobe-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465751    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.465758    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465769    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-iptables-alerter-script\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465780    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-config\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.465824    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs podName:30ad9387-03e3-49d0-8ff8-b4959c6f343f nodeName:}" failed. No retries permitted until 2026-11-04 01:54:59.965802246 +0000 UTC m=+3.097492031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs") pod "network-metrics-daemon-6g7qh" (UID: "30ad9387-03e3-49d0-8ff8-b4959c6f343f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465825    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-modprobe-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-conf-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465860    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465884    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-conf-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xjbg\" (UniqueName: \"kubernetes.io/projected/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-kube-api-access-7xjbg\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465925    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-sys\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.465965 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465961    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-system-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.465997    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466013    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466020    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-sys\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466041    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdrl6\" (UniqueName: \"kubernetes.io/projected/c54f5f65-a78e-4dd3-908f-7242d386dd33-kube-api-access-xdrl6\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466022    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466040    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-script-lib\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466077    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-host-slash\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466054    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-system-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysconfig\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466116    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-host-slash\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-kubelet\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466173    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-systemd-units\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466174    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysconfig\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466195    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-systemd-units\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.466706 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466220    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-kubelet\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-cnibin\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466281    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-config\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466291    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8g55\" (UniqueName: \"kubernetes.io/projected/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-kube-api-access-g8g55\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-log-socket\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466476    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-log-socket\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87120866-99b9-46d6-9883-6244b38c38ae-ovn-node-metrics-cert\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cni-binary-copy\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-hostroot\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466572    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-registration-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466599    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466613    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-hostroot\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8k7f\" (UniqueName: \"kubernetes.io/projected/30ad9387-03e3-49d0-8ff8-b4959c6f343f-kube-api-access-m8k7f\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-conf\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-lib-modules\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-tmp\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466756    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b94155ab-70f0-4efa-a01c-7303db0d8cb5-serviceca\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.467576 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466770    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-systemd\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-ovn\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466806    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-conf\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-socket-dir-parent\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466833    2581 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466842    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-multus\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466856    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-ovn\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466872    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-device-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466911    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-socket-dir-parent\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcwq8\" (UniqueName: \"kubernetes.io/projected/b94155ab-70f0-4efa-a01c-7303db0d8cb5-kube-api-access-zcwq8\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-tmp-dir\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466902    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-multus\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-run\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.466989    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-lib-modules\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kchqw\" (UniqueName: \"kubernetes.io/projected/87120866-99b9-46d6-9883-6244b38c38ae-kube-api-access-kchqw\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467040    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-systemd\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467043    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-k8s-cni-cncf-io\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.468265 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467083    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-k8s-cni-cncf-io\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-kubelet\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467122    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-socket-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467131    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-run\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-kubernetes\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-systemd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467225    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b94155ab-70f0-4efa-a01c-7303db0d8cb5-serviceca\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-multus-certs\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-kubelet-config\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467273    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-systemd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467282    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-multus-certs\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.467231    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467319    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b94155ab-70f0-4efa-a01c-7303db0d8cb5-host\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b94155ab-70f0-4efa-a01c-7303db0d8cb5-host\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467335    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-kubelet-config\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467336    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-tmp-dir\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467359    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-kubernetes\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.468824 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467373    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-kubelet\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.467351    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret podName:51998d39-62ca-432f-bac3-3b8142d05e0e nodeName:}" failed. No retries permitted until 2026-11-04 01:54:59.967334305 +0000 UTC m=+3.099024070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret") pod "global-pull-secret-syncer-dlxgx" (UID: "51998d39-62ca-432f-bac3-3b8142d05e0e") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-etc-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-env-overrides\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-sys-fs\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-dbus\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467528    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-etc-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467563    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-netns\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467593    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px9wr\" (UniqueName: \"kubernetes.io/projected/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-kube-api-access-px9wr\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-var-lib-kubelet\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467643    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-netns\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcrk5\" (UniqueName: \"kubernetes.io/projected/1ca946c4-34f5-4443-9ecf-e634055dacc0-kube-api-access-mcrk5\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467662    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-dbus\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-netd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467711    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-bin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-hosts-file\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-host\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.469312 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-slash\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-var-lib-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-node-log\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467850    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-hosts-file\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467872    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-env-overrides\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467854    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-slash\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467892    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-host\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467910    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-var-lib-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467923    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-netd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467865    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cnibin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467944    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-node-log\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467952    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467912    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cnibin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467953    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-bin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467820    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-var-lib-kubelet\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.467969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjg5x\" (UniqueName: \"kubernetes.io/projected/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kube-api-access-gjg5x\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.468016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.469829 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.468041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-tuned\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.470430 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.468058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-daemon-config\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.470430 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.468165    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.470430 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.470396    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-tuned\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.470430 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.470424    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-tmp\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.470612 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.470550    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87120866-99b9-46d6-9883-6244b38c38ae-ovn-node-metrics-cert\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.489678 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.489653    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 01:54:59.489773 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.489715    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d6ksr"
Nov 04 01:54:59.498201 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498174    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zk98g\""
Nov 04 01:54:59.498311 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498287    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 01:54:59.498376 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498287    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s4jjj\""
Nov 04 01:54:59.498376 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498318    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zhvgj\""
Nov 04 01:54:59.498461 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498374    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tlvkz\""
Nov 04 01:54:59.498461 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498390    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7qkcb\""
Nov 04 01:54:59.498461 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498402    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 01:54:59.498636 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498566    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bxxxw\""
Nov 04 01:54:59.498636 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498575    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bzrbt\""
Nov 04 01:54:59.498744 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498727    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 01:54:59.498793 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498761    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 01:54:59.498793 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498762    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 01:54:59.498793 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498778    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 01:54:59.498921 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498798    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 01:54:59.498921 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498812    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 01:54:59.498921 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498813    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 01:54:59.498921 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498836    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 01:54:59.498921 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498826    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 01:54:59.498921 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498871    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 01:54:59.498921 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498889    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:54:59.498921 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498913    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 01:54:59.499210 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.498939    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 01:54:59.506852 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.506826    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfca312e-b151-4c3a-99bd-dd820925264b-konnectivity-ca\") pod \"konnectivity-agent-5v5x7\" (UID: \"dfca312e-b151-4c3a-99bd-dd820925264b\") " pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:54:59.507142 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.507121    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cni-binary-copy\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.508465 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.508444    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfca312e-b151-4c3a-99bd-dd820925264b-agent-certs\") pod \"konnectivity-agent-5v5x7\" (UID: \"dfca312e-b151-4c3a-99bd-dd820925264b\") " pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:54:59.508558 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.508484    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-daemon-config\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.530082 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.530057    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:54:59.530082 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.530077    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:54:59.530234 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.530089    2581 projected.go:196] Error preparing data for projected volume kube-api-access-ptdph for pod openshift-network-diagnostics/network-check-target-jnspf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:54:59.530234 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.530183    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph podName:534103f4-2d9e-4bd7-ba78-d5e848d3924c nodeName:}" failed. No retries permitted until 2026-11-04 01:55:00.030162611 +0000 UTC m=+3.161852394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ptdph" (UniqueName: "kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph") pod "network-check-target-jnspf" (UID: "534103f4-2d9e-4bd7-ba78-d5e848d3924c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:54:59.532922 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.532875    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kchqw\" (UniqueName: \"kubernetes.io/projected/87120866-99b9-46d6-9883-6244b38c38ae-kube-api-access-kchqw\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.533292 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.533273    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcwq8\" (UniqueName: \"kubernetes.io/projected/b94155ab-70f0-4efa-a01c-7303db0d8cb5-kube-api-access-zcwq8\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.533370 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.533289    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8g55\" (UniqueName: \"kubernetes.io/projected/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-kube-api-access-g8g55\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.533370 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.533311    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xjbg\" (UniqueName: \"kubernetes.io/projected/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-kube-api-access-7xjbg\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.533445 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.533374    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8k7f\" (UniqueName: \"kubernetes.io/projected/30ad9387-03e3-49d0-8ff8-b4959c6f343f-kube-api-access-m8k7f\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:54:59.533567 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.533551    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-px9wr\" (UniqueName: \"kubernetes.io/projected/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-kube-api-access-px9wr\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.533630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.533612    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcrk5\" (UniqueName: \"kubernetes.io/projected/1ca946c4-34f5-4443-9ecf-e634055dacc0-kube-api-access-mcrk5\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.569167 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569218 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.569218 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569188    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdrl6\" (UniqueName: \"kubernetes.io/projected/c54f5f65-a78e-4dd3-908f-7242d386dd33-kube-api-access-xdrl6\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.569218 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-cnibin\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.569349 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-registration-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569349 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.569349 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-device-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569349 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569311    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-cnibin\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.569349 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569313    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569349 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-socket-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569354    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-registration-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569394    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-device-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-sys-fs\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569458    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-sys-fs\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569468    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-socket-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjg5x\" (UniqueName: \"kubernetes.io/projected/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kube-api-access-gjg5x\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-binary-copy\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569593    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-system-cni-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569599    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.569630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.570181 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569653    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-system-cni-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.570181 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569782    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.570181 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569839    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.570181 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.569835    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.570181 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.570096    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-binary-copy\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.635446 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.635414    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdrl6\" (UniqueName: \"kubernetes.io/projected/c54f5f65-a78e-4dd3-908f-7242d386dd33-kube-api-access-xdrl6\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.635580 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.635490    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjg5x\" (UniqueName: \"kubernetes.io/projected/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kube-api-access-gjg5x\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.643100 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.643080    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:54:59.653618 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:59.653586    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf00c2a_249a_4bc8_af8d_89d2f080eb16.slice/crio-ec2580eb977c12486b4e4f186385502f4763f9e61963fb08996c490edba872bf WatchSource:0}: Error finding container ec2580eb977c12486b4e4f186385502f4763f9e61963fb08996c490edba872bf: Status 404 returned error can't find the container with id ec2580eb977c12486b4e4f186385502f4763f9e61963fb08996c490edba872bf
Nov 04 01:54:59.661047 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.661032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:54:59.665045 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.665030    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mgw25"
Nov 04 01:54:59.670029 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.670005    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:54:59.670246 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:59.670225    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94155ab_70f0_4efa_a01c_7303db0d8cb5.slice/crio-0933120d5faf936a058dde92a7f114405b149969a72d2801c2b6e853f9d4260d WatchSource:0}: Error finding container 0933120d5faf936a058dde92a7f114405b149969a72d2801c2b6e853f9d4260d: Status 404 returned error can't find the container with id 0933120d5faf936a058dde92a7f114405b149969a72d2801c2b6e853f9d4260d
Nov 04 01:54:59.671445 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.671419    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:54:59.674550 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:59.674498    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4fe5f4e_25f4_400c_9efb_f2f98757c74e.slice/crio-b7fb3b328f8cc69b175873fe57227ba00e39c5a57be7129c7ad67de8d4418273 WatchSource:0}: Error finding container b7fb3b328f8cc69b175873fe57227ba00e39c5a57be7129c7ad67de8d4418273: Status 404 returned error can't find the container with id b7fb3b328f8cc69b175873fe57227ba00e39c5a57be7129c7ad67de8d4418273
Nov 04 01:54:59.675052 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.675026    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:54:59.680000 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:59.679975    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87120866_99b9_46d6_9883_6244b38c38ae.slice/crio-66634df91a7cd64cbc53723f4403b81d22922a9c62869f8cb686560f3a92f823 WatchSource:0}: Error finding container 66634df91a7cd64cbc53723f4403b81d22922a9c62869f8cb686560f3a92f823: Status 404 returned error can't find the container with id 66634df91a7cd64cbc53723f4403b81d22922a9c62869f8cb686560f3a92f823
Nov 04 01:54:59.681416 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.681401    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:54:59.685963 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:59.685938    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfca312e_b151_4c3a_99bd_dd820925264b.slice/crio-9961b1778e0f5bc3f822cd88426734388acf849099e6fa143c63c8a29743d13e WatchSource:0}: Error finding container 9961b1778e0f5bc3f822cd88426734388acf849099e6fa143c63c8a29743d13e: Status 404 returned error can't find the container with id 9961b1778e0f5bc3f822cd88426734388acf849099e6fa143c63c8a29743d13e
Nov 04 01:54:59.686643 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:59.686619    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddff859f4_bd6d_40b5_8a3d_fccf7e0baf67.slice/crio-e14b06a2ee640758ad0d97a0eac3923c7a94e83289be79f1a5934374965e31ca WatchSource:0}: Error finding container e14b06a2ee640758ad0d97a0eac3923c7a94e83289be79f1a5934374965e31ca: Status 404 returned error can't find the container with id e14b06a2ee640758ad0d97a0eac3923c7a94e83289be79f1a5934374965e31ca
Nov 04 01:54:59.692238 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.692214    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:54:59.692322 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:59.692272    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca946c4_34f5_4443_9ecf_e634055dacc0.slice/crio-a6f866714678a876485b5318dc4b1fb9cad131fd23111b287c75441822c33f6d WatchSource:0}: Error finding container a6f866714678a876485b5318dc4b1fb9cad131fd23111b287c75441822c33f6d: Status 404 returned error can't find the container with id a6f866714678a876485b5318dc4b1fb9cad131fd23111b287c75441822c33f6d
Nov 04 01:54:59.702538 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:59.702514    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaad0ba21_8234_4c49_9c72_183b7b1dbcc5.slice/crio-176cec00bd80a2d82a631b074d61bfed927d9dea6c40bb261c9653dea5e1b48c WatchSource:0}: Error finding container 176cec00bd80a2d82a631b074d61bfed927d9dea6c40bb261c9653dea5e1b48c: Status 404 returned error can't find the container with id 176cec00bd80a2d82a631b074d61bfed927d9dea6c40bb261c9653dea5e1b48c
Nov 04 01:54:59.708732 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.708715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:54:59.722234 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:54:59.722215    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54f5f65_a78e_4dd3_908f_7242d386dd33.slice/crio-dc2a797ea8cd7857e843af4d0308e69f20e8de727faa0c34048bfab6331b9160 WatchSource:0}: Error finding container dc2a797ea8cd7857e843af4d0308e69f20e8de727faa0c34048bfab6331b9160: Status 404 returned error can't find the container with id dc2a797ea8cd7857e843af4d0308e69f20e8de727faa0c34048bfab6331b9160
Nov 04 01:54:59.972651 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.972410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:54:59.972651 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:54:59.972503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:54:59.972651 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.972573    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:59.972651 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.972629    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:54:59.972985 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.972672    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs podName:30ad9387-03e3-49d0-8ff8-b4959c6f343f nodeName:}" failed. No retries permitted until 2026-11-04 01:55:00.972650195 +0000 UTC m=+4.104339960 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs") pod "network-metrics-daemon-6g7qh" (UID: "30ad9387-03e3-49d0-8ff8-b4959c6f343f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:54:59.972985 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:54:59.972700    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret podName:51998d39-62ca-432f-bac3-3b8142d05e0e nodeName:}" failed. No retries permitted until 2026-11-04 01:55:00.972682126 +0000 UTC m=+4.104371911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret") pod "global-pull-secret-syncer-dlxgx" (UID: "51998d39-62ca-432f-bac3-3b8142d05e0e") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:00.073387 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.073350    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:00.073596 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:00.073523    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:55:00.073596 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:00.073563    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:55:00.073596 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:00.073576    2581 projected.go:196] Error preparing data for projected volume kube-api-access-ptdph for pod openshift-network-diagnostics/network-check-target-jnspf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:00.073771 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:00.073647    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph podName:534103f4-2d9e-4bd7-ba78-d5e848d3924c nodeName:}" failed. No retries permitted until 2026-11-04 01:55:01.073627119 +0000 UTC m=+4.205316900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptdph" (UniqueName: "kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph") pod "network-check-target-jnspf" (UID: "534103f4-2d9e-4bd7-ba78-d5e848d3924c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:00.335772 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.335679    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c76wf" event={"ID":"c54f5f65-a78e-4dd3-908f-7242d386dd33","Type":"ContainerStarted","Data":"dc2a797ea8cd7857e843af4d0308e69f20e8de727faa0c34048bfab6331b9160"}
Nov 04 01:55:00.337019 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.336984    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4k7bb" event={"ID":"1ca946c4-34f5-4443-9ecf-e634055dacc0","Type":"ContainerStarted","Data":"a6f866714678a876485b5318dc4b1fb9cad131fd23111b287c75441822c33f6d"}
Nov 04 01:55:00.338590 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.338553    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn" event={"ID":"87120866-99b9-46d6-9883-6244b38c38ae","Type":"ContainerStarted","Data":"66634df91a7cd64cbc53723f4403b81d22922a9c62869f8cb686560f3a92f823"}
Nov 04 01:55:00.339893 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.339866    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mgw25" event={"ID":"f4fe5f4e-25f4-400c-9efb-f2f98757c74e","Type":"ContainerStarted","Data":"b7fb3b328f8cc69b175873fe57227ba00e39c5a57be7129c7ad67de8d4418273"}
Nov 04 01:55:00.341178 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.341150    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rx5p4" event={"ID":"bdf00c2a-249a-4bc8-af8d-89d2f080eb16","Type":"ContainerStarted","Data":"ec2580eb977c12486b4e4f186385502f4763f9e61963fb08996c490edba872bf"}
Nov 04 01:55:00.343151 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.343052    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm" event={"ID":"aad0ba21-8234-4c49-9c72-183b7b1dbcc5","Type":"ContainerStarted","Data":"176cec00bd80a2d82a631b074d61bfed927d9dea6c40bb261c9653dea5e1b48c"}
Nov 04 01:55:00.344611 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.344575    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tm85n" event={"ID":"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67","Type":"ContainerStarted","Data":"e14b06a2ee640758ad0d97a0eac3923c7a94e83289be79f1a5934374965e31ca"}
Nov 04 01:55:00.346052 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.346020    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5v5x7" event={"ID":"dfca312e-b151-4c3a-99bd-dd820925264b","Type":"ContainerStarted","Data":"9961b1778e0f5bc3f822cd88426734388acf849099e6fa143c63c8a29743d13e"}
Nov 04 01:55:00.347419 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.347399    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fgzvx" event={"ID":"b94155ab-70f0-4efa-a01c-7303db0d8cb5","Type":"ContainerStarted","Data":"0933120d5faf936a058dde92a7f114405b149969a72d2801c2b6e853f9d4260d"}
Nov 04 01:55:00.494448 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.494375    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:49:59 +0000 UTC" deadline="2028-05-13 23:42:29.378725442 +0000 UTC"
Nov 04 01:55:00.494448 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.494440    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13365h47m28.884289937s"
Nov 04 01:55:00.982521 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.982478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:00.982739 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:00.982576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:00.982739 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:00.982697    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:00.982859 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:00.982757    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs podName:30ad9387-03e3-49d0-8ff8-b4959c6f343f nodeName:}" failed. No retries permitted until 2026-11-04 01:55:02.98273746 +0000 UTC m=+6.114427239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs") pod "network-metrics-daemon-6g7qh" (UID: "30ad9387-03e3-49d0-8ff8-b4959c6f343f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:00.982859 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:00.982827    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:00.982998 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:00.982864    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret podName:51998d39-62ca-432f-bac3-3b8142d05e0e nodeName:}" failed. No retries permitted until 2026-11-04 01:55:02.982851949 +0000 UTC m=+6.114541718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret") pod "global-pull-secret-syncer-dlxgx" (UID: "51998d39-62ca-432f-bac3-3b8142d05e0e") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:01.083514 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:01.083473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:01.083759 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:01.083741    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:55:01.083833 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:01.083768    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:55:01.083833 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:01.083785    2581 projected.go:196] Error preparing data for projected volume kube-api-access-ptdph for pod openshift-network-diagnostics/network-check-target-jnspf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:01.083939 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:01.083850    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph podName:534103f4-2d9e-4bd7-ba78-d5e848d3924c nodeName:}" failed. No retries permitted until 2026-11-04 01:55:03.083831018 +0000 UTC m=+6.215520785 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptdph" (UniqueName: "kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph") pod "network-check-target-jnspf" (UID: "534103f4-2d9e-4bd7-ba78-d5e848d3924c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:01.330794 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:01.330748    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:01.330947 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:01.330886    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:01.331022 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:01.331005    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:01.331130 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:01.331110    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:01.331193 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:01.331180    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:01.331277 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:01.331258    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:01.495648 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:01.495604    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:49:59 +0000 UTC" deadline="2028-06-04 01:49:39.704378993 +0000 UTC"
Nov 04 01:55:01.495648 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:01.495646    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13871h54m38.208738438s"
Nov 04 01:55:02.390396 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:02.390170    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"23463ee86a57f13d80b429c8e67d6e17","Type":"ContainerStarted","Data":"746449b54e251e758b30d72993e52f67dc5f0bbeaa295b28b6d70fd85ffeee42"}
Nov 04 01:55:03.003002 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.002961    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:03.003500 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.003046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:03.003500 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.003183    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:03.003500 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.003247    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs podName:30ad9387-03e3-49d0-8ff8-b4959c6f343f nodeName:}" failed. No retries permitted until 2026-11-04 01:55:07.003226468 +0000 UTC m=+10.134916251 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs") pod "network-metrics-daemon-6g7qh" (UID: "30ad9387-03e3-49d0-8ff8-b4959c6f343f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:03.003500 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.003317    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:03.003500 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.003356    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret podName:51998d39-62ca-432f-bac3-3b8142d05e0e nodeName:}" failed. No retries permitted until 2026-11-04 01:55:07.003345012 +0000 UTC m=+10.135034778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret") pod "global-pull-secret-syncer-dlxgx" (UID: "51998d39-62ca-432f-bac3-3b8142d05e0e") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:03.104311 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.104216    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:03.104447 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.104420    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:55:03.104508 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.104452    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:55:03.104508 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.104465    2581 projected.go:196] Error preparing data for projected volume kube-api-access-ptdph for pod openshift-network-diagnostics/network-check-target-jnspf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:03.104643 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.104551    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph podName:534103f4-2d9e-4bd7-ba78-d5e848d3924c nodeName:}" failed. No retries permitted until 2026-11-04 01:55:07.104513651 +0000 UTC m=+10.236203419 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptdph" (UniqueName: "kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph") pod "network-check-target-jnspf" (UID: "534103f4-2d9e-4bd7-ba78-d5e848d3924c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:03.325907 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.325698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:03.325907 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.325848    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:03.326624 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.326270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:03.326624 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.326390    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:03.326624 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.326467    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:03.326624 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:03.326544    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:03.399184 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.398319    2581 generic.go:356] "Generic (PLEG): container finished" podID="23463ee86a57f13d80b429c8e67d6e17" containerID="746449b54e251e758b30d72993e52f67dc5f0bbeaa295b28b6d70fd85ffeee42" exitCode=0
Nov 04 01:55:03.399184 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.398758    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"23463ee86a57f13d80b429c8e67d6e17","Type":"ContainerDied","Data":"746449b54e251e758b30d72993e52f67dc5f0bbeaa295b28b6d70fd85ffeee42"}
Nov 04 01:55:03.422937 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.421955    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c76wf" event={"ID":"c54f5f65-a78e-4dd3-908f-7242d386dd33","Type":"ContainerStarted","Data":"fb676d661078966974d3101e91ed770cf479e0cdd1ab52f5ba015fb5fe85e162"}
Nov 04 01:55:03.435378 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.434822    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"9f88ddff8fbb738ea2fba74409a8de90","Type":"ContainerStarted","Data":"229db822cb3ffcd021eec334a6d9eab14f19664d43a0377399df88ef43f1f028"}
Nov 04 01:55:03.779328 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:03.779265    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal" podStartSLOduration=4.779245203 podStartE2EDuration="4.779245203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:03.778802537 +0000 UTC m=+6.910492326" watchObservedRunningTime="2026-11-04 01:55:03.779245203 +0000 UTC m=+6.910934994"
Nov 04 01:55:04.442415 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:04.442370    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"23463ee86a57f13d80b429c8e67d6e17","Type":"ContainerStarted","Data":"7300cc1c00dac93c7538156cb14e9127caefba388ef0b1ff993aac7c78d542d8"}
Nov 04 01:55:04.451913 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:04.451863    2581 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="fb676d661078966974d3101e91ed770cf479e0cdd1ab52f5ba015fb5fe85e162" exitCode=0
Nov 04 01:55:04.453112 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:04.453050    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c76wf" event={"ID":"c54f5f65-a78e-4dd3-908f-7242d386dd33","Type":"ContainerDied","Data":"fb676d661078966974d3101e91ed770cf479e0cdd1ab52f5ba015fb5fe85e162"}
Nov 04 01:55:04.569409 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:04.569337    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal" podStartSLOduration=5.569316747 podStartE2EDuration="5.569316747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:04.568245911 +0000 UTC m=+7.699935700" watchObservedRunningTime="2026-11-04 01:55:04.569316747 +0000 UTC m=+7.701006537"
Nov 04 01:55:05.325777 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:05.325745    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:05.325777 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:05.325770    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:05.325988 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:05.325912    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:05.326063 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:05.326037    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:05.326063 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:05.326053    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:05.326254 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:05.326186    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:05.464295 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:05.463287    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5v5x7" event={"ID":"dfca312e-b151-4c3a-99bd-dd820925264b","Type":"ContainerStarted","Data":"fb2ad20bb3ba21da58e20027c0c5905b88aa768fa066d0c02a7e1d42061c7687"}
Nov 04 01:55:06.474151 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:06.474027    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm" event={"ID":"aad0ba21-8234-4c49-9c72-183b7b1dbcc5","Type":"ContainerStarted","Data":"f9683a77466e7bf500efc2a2b870f715f9c1011981a956a1a187f505bef65609"}
Nov 04 01:55:06.483121 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:06.483037    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fgzvx" event={"ID":"b94155ab-70f0-4efa-a01c-7303db0d8cb5","Type":"ContainerStarted","Data":"f47a1256176fce60d477c500f20dee2b6557499ec6ee031fba71402b105073a1"}
Nov 04 01:55:06.607579 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:06.607511    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5v5x7" podStartSLOduration=3.823897504 podStartE2EDuration="8.607489904s" totalImagesPullingTime="4.7835924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:05.600619595 +0000 UTC m=+8.732309390" watchObservedRunningTime="2026-11-04 01:55:06.607489904 +0000 UTC m=+9.739179694"
Nov 04 01:55:07.038486 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:07.038448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:07.038694 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:07.038541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:07.038748 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.038727    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:07.038791 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.038782    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs podName:30ad9387-03e3-49d0-8ff8-b4959c6f343f nodeName:}" failed. No retries permitted until 2026-11-04 01:55:15.038768311 +0000 UTC m=+18.170458074 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs") pod "network-metrics-daemon-6g7qh" (UID: "30ad9387-03e3-49d0-8ff8-b4959c6f343f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:07.038851 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.038839    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:07.038886 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.038870    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret podName:51998d39-62ca-432f-bac3-3b8142d05e0e nodeName:}" failed. No retries permitted until 2026-11-04 01:55:15.038862422 +0000 UTC m=+18.170552185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret") pod "global-pull-secret-syncer-dlxgx" (UID: "51998d39-62ca-432f-bac3-3b8142d05e0e") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:07.139695 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:07.139648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:07.139879 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.139861    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:55:07.139958 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.139886    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:55:07.139958 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.139901    2581 projected.go:196] Error preparing data for projected volume kube-api-access-ptdph for pod openshift-network-diagnostics/network-check-target-jnspf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:07.140047 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.139965    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph podName:534103f4-2d9e-4bd7-ba78-d5e848d3924c nodeName:}" failed. No retries permitted until 2026-11-04 01:55:15.139945905 +0000 UTC m=+18.271635690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptdph" (UniqueName: "kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph") pod "network-check-target-jnspf" (UID: "534103f4-2d9e-4bd7-ba78-d5e848d3924c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:07.327282 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:07.327201    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:07.327448 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.327326    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:07.327448 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:07.327373    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:07.327448 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.327427    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:07.327668 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:07.327455    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:07.327668 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:07.327503    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:08.013314 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:08.013274    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:55:08.014221 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:08.014184    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:55:08.128895 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:08.128626    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fgzvx" podStartSLOduration=4.186613505 podStartE2EDuration="10.128610651s" totalImagesPullingTime="5.941997146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:06.607981383 +0000 UTC m=+9.739671176" watchObservedRunningTime="2026-11-04 01:55:08.128610651 +0000 UTC m=+11.260300433"
Nov 04 01:55:08.703141 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:08.703101    2581 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 01:55:09.323263 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.323152    2581 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:55:08.703125413Z","UUID":"68297152-161f-43b1-bbd2-f40eeed1e145","Handler":null,"Name":"","Endpoint":""}
Nov 04 01:55:09.324856 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.324836    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 01:55:09.324956 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.324866    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 01:55:09.325371 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.325344    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:09.325371 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.325365    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:09.325491 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:09.325462    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:09.325595 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.325572    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:09.325725 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:09.325697    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:09.325841 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:09.325811    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:09.363051 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.363029    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:55:09.363670 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.363639    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:55:09.495755 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.495720    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm" event={"ID":"aad0ba21-8234-4c49-9c72-183b7b1dbcc5","Type":"ContainerStarted","Data":"20b1c6f6f824225acb66f344914ad27c5bf041a6d939a754c6861b1d8abbbc9d"}
Nov 04 01:55:09.497442 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.497388    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tm85n" event={"ID":"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67","Type":"ContainerStarted","Data":"8a822ff0b9e14b3bad16900bb49eed1e0f6da9f093a8dd46bf1595af43fa69c4"}
Nov 04 01:55:09.499585 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.499556    2581 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="67f262fbffd17fbf962e574290519da9573fed286dfabcd5dd62360e753fc3a3" exitCode=0
Nov 04 01:55:09.499673 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.499640    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c76wf" event={"ID":"c54f5f65-a78e-4dd3-908f-7242d386dd33","Type":"ContainerDied","Data":"67f262fbffd17fbf962e574290519da9573fed286dfabcd5dd62360e753fc3a3"}
Nov 04 01:55:09.501881 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.501857    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mgw25" event={"ID":"f4fe5f4e-25f4-400c-9efb-f2f98757c74e","Type":"ContainerStarted","Data":"b9a5469ce289ea765792d689e49b060f349123b849b6ab60928522713768b54e"}
Nov 04 01:55:09.613568 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.613254    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-tm85n" podStartSLOduration=2.301794065 podStartE2EDuration="11.613235063s" totalImagesPullingTime="9.311440998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:09.612315749 +0000 UTC m=+12.744005534" watchObservedRunningTime="2026-11-04 01:55:09.613235063 +0000 UTC m=+12.744924848"
Nov 04 01:55:09.854837 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:09.854786    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-mgw25" podStartSLOduration=3.084667318 podStartE2EDuration="11.854770601s" totalImagesPullingTime="8.770103283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:09.854128665 +0000 UTC m=+12.985818452" watchObservedRunningTime="2026-11-04 01:55:09.854770601 +0000 UTC m=+12.986460385"
Nov 04 01:55:10.506471 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:10.506424    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rx5p4" event={"ID":"bdf00c2a-249a-4bc8-af8d-89d2f080eb16","Type":"ContainerStarted","Data":"416adcf8d304edb87a63b6127a3e3ac3f4792fe11573e014e3b6c50dd388ed16"}
Nov 04 01:55:11.325834 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:11.325753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:11.325834 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:11.325795    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:11.326037 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:11.325872    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:11.326037 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:11.325887    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:11.326037 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:11.325979    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:11.326158 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:11.326065    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:11.511313 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:11.511263    2581 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="3d33afd7a73fc81601d301d8c409668cdf4c50e29636482ffed9e4840d34f4e0" exitCode=0
Nov 04 01:55:11.511960 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:11.511346    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c76wf" event={"ID":"c54f5f65-a78e-4dd3-908f-7242d386dd33","Type":"ContainerDied","Data":"3d33afd7a73fc81601d301d8c409668cdf4c50e29636482ffed9e4840d34f4e0"}
Nov 04 01:55:11.515551 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:11.515502    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4k7bb" event={"ID":"1ca946c4-34f5-4443-9ecf-e634055dacc0","Type":"ContainerStarted","Data":"b2613d3ae5499e1b4ebca599d59b1a93583db7c91fa681b5319db5b53efcf9a9"}
Nov 04 01:55:11.518570 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:11.518486    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm" event={"ID":"aad0ba21-8234-4c49-9c72-183b7b1dbcc5","Type":"ContainerStarted","Data":"8f3beb0273926c274f5c5318fc7065f24ddc9b35fa7f897df3217eb0cd1321c3"}
Nov 04 01:55:11.633821 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:11.633727    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rx5p4" podStartSLOduration=4.590763143 podStartE2EDuration="13.63370957s" totalImagesPullingTime="9.042946427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:10.622700682 +0000 UTC m=+13.754390649" watchObservedRunningTime="2026-11-04 01:55:11.63370957 +0000 UTC m=+14.765399355"
Nov 04 01:55:11.751153 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:11.751094    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4k7bb" podStartSLOduration=2.420341568 podStartE2EDuration="13.751076184s" totalImagesPullingTime="11.330734616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:11.750859522 +0000 UTC m=+14.882549309" watchObservedRunningTime="2026-11-04 01:55:11.751076184 +0000 UTC m=+14.882765968"
Nov 04 01:55:11.869703 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:11.869644    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm" podStartSLOduration=3.442294463 podStartE2EDuration="13.869630063s" totalImagesPullingTime="10.4273356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:11.868921803 +0000 UTC m=+15.000611586" watchObservedRunningTime="2026-11-04 01:55:11.869630063 +0000 UTC m=+15.001319847"
Nov 04 01:55:13.325552 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:13.325520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:13.326177 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:13.325602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:13.326177 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:13.325719    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:13.326177 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:13.325736    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:13.326177 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:13.325825    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:13.326177 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:13.325890    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:13.526177 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:13.526135    2581 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="2ac8b1900815acec0d409e3960e8cde0f5cab35ec3f6bf010afff9802ebed08e" exitCode=0
Nov 04 01:55:13.526334 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:13.526216    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c76wf" event={"ID":"c54f5f65-a78e-4dd3-908f-7242d386dd33","Type":"ContainerDied","Data":"2ac8b1900815acec0d409e3960e8cde0f5cab35ec3f6bf010afff9802ebed08e"}
Nov 04 01:55:15.102395 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:15.102357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:15.102862 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:15.102421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:15.102862 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.102523    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:15.102862 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.102523    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:15.102862 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.102590    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs podName:30ad9387-03e3-49d0-8ff8-b4959c6f343f nodeName:}" failed. No retries permitted until 2026-11-04 01:55:31.102571746 +0000 UTC m=+34.234261508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs") pod "network-metrics-daemon-6g7qh" (UID: "30ad9387-03e3-49d0-8ff8-b4959c6f343f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:55:15.102862 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.102604    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret podName:51998d39-62ca-432f-bac3-3b8142d05e0e nodeName:}" failed. No retries permitted until 2026-11-04 01:55:31.102598398 +0000 UTC m=+34.234288160 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret") pod "global-pull-secret-syncer-dlxgx" (UID: "51998d39-62ca-432f-bac3-3b8142d05e0e") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:55:15.203794 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:15.203755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:15.203965 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.203905    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:55:15.203965 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.203929    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:55:15.203965 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.203943    2581 projected.go:196] Error preparing data for projected volume kube-api-access-ptdph for pod openshift-network-diagnostics/network-check-target-jnspf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:15.204095 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.204007    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph podName:534103f4-2d9e-4bd7-ba78-d5e848d3924c nodeName:}" failed. No retries permitted until 2026-11-04 01:55:31.203993767 +0000 UTC m=+34.335683529 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptdph" (UniqueName: "kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph") pod "network-check-target-jnspf" (UID: "534103f4-2d9e-4bd7-ba78-d5e848d3924c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:55:15.325905 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:15.325868    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:15.326063 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:15.325913    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:15.326063 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:15.325873    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:15.326063 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.326017    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:15.326216 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.326099    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:15.326216 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:15.326199    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:15.532475 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:15.532355    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn" event={"ID":"87120866-99b9-46d6-9883-6244b38c38ae","Type":"ContainerStarted","Data":"21216a1f68f3350632520947e2145c105da1935682b8038c69310fb50a29e194"}
Nov 04 01:55:16.536993 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:16.536752    2581 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="d1433ef44defdfb7ae5aa38233ae7c474d6ef851a09581871a12a103c4e9f0a2" exitCode=0
Nov 04 01:55:16.537441 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:16.536825    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c76wf" event={"ID":"c54f5f65-a78e-4dd3-908f-7242d386dd33","Type":"ContainerDied","Data":"d1433ef44defdfb7ae5aa38233ae7c474d6ef851a09581871a12a103c4e9f0a2"}
Nov 04 01:55:16.540282 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:16.540262    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn" event={"ID":"87120866-99b9-46d6-9883-6244b38c38ae","Type":"ContainerStarted","Data":"0d8f7dda903cb6589b67e10f8e2dc32e1145bf84a594c71cf0050098d577a031"}
Nov 04 01:55:16.540345 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:16.540325    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn" event={"ID":"87120866-99b9-46d6-9883-6244b38c38ae","Type":"ContainerStarted","Data":"c3957d545b983834fd93b14e9ad7be812ef541b886ba11fc6a3844a5e17658d5"}
Nov 04 01:55:16.540375 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:16.540354    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn" event={"ID":"87120866-99b9-46d6-9883-6244b38c38ae","Type":"ContainerStarted","Data":"dad9baf2798e80e61904a3e5f572cbf0cacc9150fa6c5ac2180ba334249a0367"}
Nov 04 01:55:16.540409 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:16.540388    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn" event={"ID":"87120866-99b9-46d6-9883-6244b38c38ae","Type":"ContainerStarted","Data":"fd44a9fd6609af48e3853a11ed50f17f20cc11aa77244d98d5aa9bddd135a63f"}
Nov 04 01:55:16.540439 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:16.540423    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn" event={"ID":"87120866-99b9-46d6-9883-6244b38c38ae","Type":"ContainerStarted","Data":"215f54150a2520abaa6e83b6aaa1080bf1b1c71e16aa4cf089429c396d94a004"}
Nov 04 01:55:17.326561 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:17.326532    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:17.326713 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:17.326631    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:17.326713 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:17.326647    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:17.326713 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:17.326703    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:17.326863 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:17.326768    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:17.326863 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:17.326825    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:17.545377 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:17.545345    2581 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="0ea2e4889a3b8c8f4551bd4a242cef3e55edceaa8226ec81aa5c32d882aa6870" exitCode=0
Nov 04 01:55:17.545798 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:17.545393    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c76wf" event={"ID":"c54f5f65-a78e-4dd3-908f-7242d386dd33","Type":"ContainerDied","Data":"0ea2e4889a3b8c8f4551bd4a242cef3e55edceaa8226ec81aa5c32d882aa6870"}
Nov 04 01:55:18.550640 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:18.550601    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c76wf" event={"ID":"c54f5f65-a78e-4dd3-908f-7242d386dd33","Type":"ContainerStarted","Data":"fef161f13519faa933e4da934a1f1773284ebe32c31a3a9b786014130c71f677"}
Nov 04 01:55:18.553788 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:18.553762    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn" event={"ID":"87120866-99b9-46d6-9883-6244b38c38ae","Type":"ContainerStarted","Data":"7995474d54144bf67d0549f8c013016420fcf3eeb12e382872b91f3aa13f2e70"}
Nov 04 01:55:18.673183 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:18.673132    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-c76wf" podStartSLOduration=7.979997173 podStartE2EDuration="20.673120229s" totalImagesPullingTime="12.693123056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:18.672480996 +0000 UTC m=+21.804170780" watchObservedRunningTime="2026-11-04 01:55:18.673120229 +0000 UTC m=+21.804810012"
Nov 04 01:55:19.325995 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:19.325963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:19.326183 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:19.325963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:19.326183 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:19.326079    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:19.326183 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:19.326071    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:19.326183 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:19.326153    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:19.326349 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:19.326258    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:20.561676 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:20.561643    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn" event={"ID":"87120866-99b9-46d6-9883-6244b38c38ae","Type":"ContainerStarted","Data":"5158b5a015e96066882756fe8e9a25aae901d05babd550ae543b38e47d0b632a"}
Nov 04 01:55:20.562454 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:20.561933    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:20.562454 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:20.562007    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:20.578431 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:20.578405    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:20.839802 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:20.839553    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn" podStartSLOduration=7.106672823 podStartE2EDuration="22.83953761s" totalImagesPullingTime="15.732864787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:20.789210144 +0000 UTC m=+23.920899931" watchObservedRunningTime="2026-11-04 01:55:20.83953761 +0000 UTC m=+23.971227394"
Nov 04 01:55:21.326130 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:21.326093    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:21.326335 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:21.326104    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:21.326335 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:21.326253    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:21.326335 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:21.326301    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:21.326493 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:21.326394    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:21.326493 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:21.326466    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:21.563808 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:21.563780    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:21.585690 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:21.585616    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:22.601406 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:22.601336    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dlxgx"]
Nov 04 01:55:22.601851 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:22.601451    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:22.601851 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:22.601581    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:22.602093 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:22.602076    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jnspf"]
Nov 04 01:55:22.602159 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:22.602147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:22.602237 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:22.602220    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:22.603593 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:22.603569    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6g7qh"]
Nov 04 01:55:22.603688 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:22.603649    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:22.603735 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:22.603724    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:23.066320 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:23.066282    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:23.319470 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:23.319396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:23.523010 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:23.522979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:23.789367 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:23.789334    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:23.991027 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:23.990997    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:24.325922 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:24.325842    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:24.325922 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:24.325869    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:24.325922 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:24.325865    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:24.326106 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:24.325954    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:24.326106 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:24.326032    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:24.326183 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:24.326104    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:26.234900 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:26.234863    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:26.325618 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:26.325592    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:26.325775 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:26.325688    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:26.325775 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:26.325705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:26.325854 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:26.325789    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:26.325854 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:26.325710    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:26.325920 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:26.325858    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:26.446685 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:26.446655    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:27.260240 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:27.260208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:27.460203 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:27.460174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:28.326445 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.326229    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:28.326789 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.326261    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:28.326789 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:28.326464    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jnspf" podUID="534103f4-2d9e-4bd7-ba78-d5e848d3924c"
Nov 04 01:55:28.326789 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.326296    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:28.326789 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:28.326591    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dlxgx" podUID="51998d39-62ca-432f-bac3-3b8142d05e0e"
Nov 04 01:55:28.326789 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:28.326634    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6g7qh" podUID="30ad9387-03e3-49d0-8ff8-b4959c6f343f"
Nov 04 01:55:28.622442 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.622362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:28.653131 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.653109    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeReady"
Nov 04 01:55:28.653257 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.653223    2581 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 01:55:28.718267 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.718246    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeReady"
Nov 04 01:55:28.768559 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.768530    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l89qg"]
Nov 04 01:55:28.771580 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.771563    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qpbh2"]
Nov 04 01:55:28.771700 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.771684    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:28.774146 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.774130    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:28.824528 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.824471    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 01:55:28.824683 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.824592    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 01:55:28.824683 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.824619    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k89rs\""
Nov 04 01:55:28.828466 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.828443    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 01:55:28.829820 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.829799    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qpbh2"]
Nov 04 01:55:28.832977 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.832953    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l89qg"]
Nov 04 01:55:28.834154 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.834130    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:28.897537 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.897451    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl6p9\" (UniqueName: \"kubernetes.io/projected/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-kube-api-access-vl6p9\") pod \"ingress-canary-qpbh2\" (UID: \"d91062aa-c9b2-48ac-865a-fc0acb25b4a6\") " pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:28.897537 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.897484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjzd4\" (UniqueName: \"kubernetes.io/projected/eb7f58fe-db82-4c9e-bdfd-39725432e053-kube-api-access-hjzd4\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:28.897697 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.897605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb7f58fe-db82-4c9e-bdfd-39725432e053-config-volume\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:28.897697 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.897630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-cert\") pod \"ingress-canary-qpbh2\" (UID: \"d91062aa-c9b2-48ac-865a-fc0acb25b4a6\") " pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:28.897697 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.897648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:28.897697 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.897661    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb7f58fe-db82-4c9e-bdfd-39725432e053-tmp-dir\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:28.898114 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.898097    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 01:55:28.898193 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.898179    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-9ldb4\""
Nov 04 01:55:28.901592 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.901576    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 01:55:28.998856 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.998826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjzd4\" (UniqueName: \"kubernetes.io/projected/eb7f58fe-db82-4c9e-bdfd-39725432e053-kube-api-access-hjzd4\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:28.998944 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.998879    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb7f58fe-db82-4c9e-bdfd-39725432e053-config-volume\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:28.999078 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.999055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-cert\") pod \"ingress-canary-qpbh2\" (UID: \"d91062aa-c9b2-48ac-865a-fc0acb25b4a6\") " pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:28.999111 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.999098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:28.999155 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.999129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb7f58fe-db82-4c9e-bdfd-39725432e053-tmp-dir\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:28.999189 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:28.999173    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:28.999189 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.999182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl6p9\" (UniqueName: \"kubernetes.io/projected/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-kube-api-access-vl6p9\") pod \"ingress-canary-qpbh2\" (UID: \"d91062aa-c9b2-48ac-865a-fc0acb25b4a6\") " pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:28.999271 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:28.999222    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls podName:eb7f58fe-db82-4c9e-bdfd-39725432e053 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:29.499208466 +0000 UTC m=+32.630898227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls") pod "dns-default-l89qg" (UID: "eb7f58fe-db82-4c9e-bdfd-39725432e053") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:28.999477 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.999456    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb7f58fe-db82-4c9e-bdfd-39725432e053-tmp-dir\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:28.999574 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:28.999496    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb7f58fe-db82-4c9e-bdfd-39725432e053-config-volume\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:29.003539 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.003494    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-cert\") pod \"ingress-canary-qpbh2\" (UID: \"d91062aa-c9b2-48ac-865a-fc0acb25b4a6\") " pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:29.062386 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.062355    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl6p9\" (UniqueName: \"kubernetes.io/projected/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-kube-api-access-vl6p9\") pod \"ingress-canary-qpbh2\" (UID: \"d91062aa-c9b2-48ac-865a-fc0acb25b4a6\") " pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:29.063130 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.063107    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjzd4\" (UniqueName: \"kubernetes.io/projected/eb7f58fe-db82-4c9e-bdfd-39725432e053-kube-api-access-hjzd4\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:29.089172 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.089145    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:29.465680 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.465312    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qpbh2"]
Nov 04 01:55:29.469388 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:55:29.469356    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91062aa_c9b2_48ac_865a_fc0acb25b4a6.slice/crio-a6dc01b4f280c69b31a786329af3165111eb7fbc4cbe6fc98d352341a87ad56c WatchSource:0}: Error finding container a6dc01b4f280c69b31a786329af3165111eb7fbc4cbe6fc98d352341a87ad56c: Status 404 returned error can't find the container with id a6dc01b4f280c69b31a786329af3165111eb7fbc4cbe6fc98d352341a87ad56c
Nov 04 01:55:29.503162 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.503137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:29.503301 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:29.503284    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:29.503352 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:29.503343    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls podName:eb7f58fe-db82-4c9e-bdfd-39725432e053 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.503326611 +0000 UTC m=+33.635016389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls") pod "dns-default-l89qg" (UID: "eb7f58fe-db82-4c9e-bdfd-39725432e053") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:29.579967 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.579940    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qpbh2" event={"ID":"d91062aa-c9b2-48ac-865a-fc0acb25b4a6","Type":"ContainerStarted","Data":"a6dc01b4f280c69b31a786329af3165111eb7fbc4cbe6fc98d352341a87ad56c"}
Nov 04 01:55:29.657887 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.657857    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s2nfk"]
Nov 04 01:55:29.661910 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.661891    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.714367 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.714345    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 01:55:29.714600 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.714578    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mj9m5\""
Nov 04 01:55:29.714673 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.714620    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 01:55:29.714673 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.714620    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 01:55:29.714673 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.714621    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 01:55:29.719854 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.719797    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s2nfk"]
Nov 04 01:55:29.807666 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.807631    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e882a774-f295-43dd-916c-e20c2df7f05a-data-volume\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.807816 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.807759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.807816 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.807802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e882a774-f295-43dd-916c-e20c2df7f05a-crio-socket\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.807922 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.807825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zzbc\" (UniqueName: \"kubernetes.io/projected/e882a774-f295-43dd-916c-e20c2df7f05a-kube-api-access-4zzbc\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.807922 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.807858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.908960 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.908928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.909070 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.908973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e882a774-f295-43dd-916c-e20c2df7f05a-crio-socket\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.909070 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.908994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zzbc\" (UniqueName: \"kubernetes.io/projected/e882a774-f295-43dd-916c-e20c2df7f05a-kube-api-access-4zzbc\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.909070 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.909022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.909070 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.909046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e882a774-f295-43dd-916c-e20c2df7f05a-data-volume\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.909247 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:29.909109    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:55:29.909247 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:29.909107    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:29.909247 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:29.909162    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.40914911 +0000 UTC m=+33.540838884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : configmap "kube-rbac-proxy" not found
Nov 04 01:55:29.909247 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.909162    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e882a774-f295-43dd-916c-e20c2df7f05a-crio-socket\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.909247 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:29.909173    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.409167999 +0000 UTC m=+33.540857760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:29.909473 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.909428    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e882a774-f295-43dd-916c-e20c2df7f05a-data-volume\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.920466 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.920442    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-lzc9g-h2w8g"]
Nov 04 01:55:29.923303 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.923289    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:29.950656 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.950632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:29.972212 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.972154    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zzbc\" (UniqueName: \"kubernetes.io/projected/e882a774-f295-43dd-916c-e20c2df7f05a-kube-api-access-4zzbc\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:29.975993 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.975974    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6wh47\""
Nov 04 01:55:29.975993 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:29.975974    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 01:55:30.010125 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.010091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e7d3eb2e-9765-455d-b901-6596bdb56f7e-archives-path\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.010251 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.010129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9k72\" (UniqueName: \"kubernetes.io/projected/e7d3eb2e-9765-455d-b901-6596bdb56f7e-kube-api-access-r9k72\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.010301 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.010279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-trusted-ca-bundle\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.010337 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.010319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-service-ca-bundle\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.070817 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.070784    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 01:55:30.071540 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.071519    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-lzc9g-h2w8g"]
Nov 04 01:55:30.110678 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.110649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-service-ca-bundle\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.110813 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.110710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e7d3eb2e-9765-455d-b901-6596bdb56f7e-archives-path\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.110813 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.110746    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9k72\" (UniqueName: \"kubernetes.io/projected/e7d3eb2e-9765-455d-b901-6596bdb56f7e-kube-api-access-r9k72\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.110880 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.110820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-trusted-ca-bundle\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.111170 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.111152    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e7d3eb2e-9765-455d-b901-6596bdb56f7e-archives-path\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.111391 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.111368    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-service-ca-bundle\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.111579 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.111564    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-trusted-ca-bundle\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.151149 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.151119    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:30.204715 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.204678    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9k72\" (UniqueName: \"kubernetes.io/projected/e7d3eb2e-9765-455d-b901-6596bdb56f7e-kube-api-access-r9k72\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.233521 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.233492    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:30.326204 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.326168    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:30.326348 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.326245    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:30.326348 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.326331    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:30.379427 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.379398    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:55:30.379600 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.379550    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:55:30.379600 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.379551    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:55:30.379600 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.379590    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tvbnh\""
Nov 04 01:55:30.379720 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.379619    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wpjt7\""
Nov 04 01:55:30.379720 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.379641    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zzzll\""
Nov 04 01:55:30.380346 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.380333    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:55:30.412670 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.412647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:30.412792 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.412708    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:30.412849 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:30.412794    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:30.412849 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:30.412808    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:55:30.412911 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:30.412853    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:31.412839611 +0000 UTC m=+34.544529375 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:30.412911 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:30.412867    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:31.412860502 +0000 UTC m=+34.544550263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : configmap "kube-rbac-proxy" not found
Nov 04 01:55:30.513926 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.513850    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:30.514354 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:30.514011    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:30.514354 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:30.514073    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls podName:eb7f58fe-db82-4c9e-bdfd-39725432e053 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:32.514055387 +0000 UTC m=+35.645745162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls") pod "dns-default-l89qg" (UID: "eb7f58fe-db82-4c9e-bdfd-39725432e053") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:30.600348 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:30.600318    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-lzc9g-h2w8g"]
Nov 04 01:55:30.604328 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:55:30.604290    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d3eb2e_9765_455d_b901_6596bdb56f7e.slice/crio-e3fcec2a2abff87025a81d319b1a140f47f7b17c6ad193209468b49363536050 WatchSource:0}: Error finding container e3fcec2a2abff87025a81d319b1a140f47f7b17c6ad193209468b49363536050: Status 404 returned error can't find the container with id e3fcec2a2abff87025a81d319b1a140f47f7b17c6ad193209468b49363536050
Nov 04 01:55:31.118776 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.118738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:31.118944 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.118839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:31.118944 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:31.118903    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:55:31.119059 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:31.118973    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs podName:30ad9387-03e3-49d0-8ff8-b4959c6f343f nodeName:}" failed. No retries permitted until 2026-11-04 01:56:03.118957415 +0000 UTC m=+66.250647186 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs") pod "network-metrics-daemon-6g7qh" (UID: "30ad9387-03e3-49d0-8ff8-b4959c6f343f") : secret "metrics-daemon-secret" not found
Nov 04 01:55:31.121554 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.121530    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:31.220235 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.220203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:31.223123 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.223095    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:31.228832 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.228809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:31.238911 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.238887    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:31.249637 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.249618    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:31.372681 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.372603    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4"]
Nov 04 01:55:31.377630 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.377605    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4"
Nov 04 01:55:31.421846 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.421809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:31.422033 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.421868    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:31.422033 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:31.421994    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:55:31.422215 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:31.422047    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:33.422032377 +0000 UTC m=+36.553722159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : configmap "kube-rbac-proxy" not found
Nov 04 01:55:31.422467 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:31.422429    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:31.422614 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:31.422483    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:33.422470127 +0000 UTC m=+36.554159902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:31.431688 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.431665    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-c7hth\""
Nov 04 01:55:31.431802 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.431786    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 01:55:31.431915 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.431896    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:31.432711 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.432690    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4"]
Nov 04 01:55:31.440891 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.440828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:31.522352 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.522309    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqmq6\" (UniqueName: \"kubernetes.io/projected/8369e5ee-521f-4922-9aff-7bcacc063cfa-kube-api-access-jqmq6\") pod \"migrator-66df5d496b-6xdt4\" (UID: \"8369e5ee-521f-4922-9aff-7bcacc063cfa\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4"
Nov 04 01:55:31.588669 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.588623    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lzc9g-h2w8g" event={"ID":"e7d3eb2e-9765-455d-b901-6596bdb56f7e","Type":"ContainerStarted","Data":"e3fcec2a2abff87025a81d319b1a140f47f7b17c6ad193209468b49363536050"}
Nov 04 01:55:31.619561 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.619522    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dlxgx"]
Nov 04 01:55:31.623090 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.623015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqmq6\" (UniqueName: \"kubernetes.io/projected/8369e5ee-521f-4922-9aff-7bcacc063cfa-kube-api-access-jqmq6\") pod \"migrator-66df5d496b-6xdt4\" (UID: \"8369e5ee-521f-4922-9aff-7bcacc063cfa\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4"
Nov 04 01:55:31.635045 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:55:31.635011    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51998d39_62ca_432f_bac3_3b8142d05e0e.slice/crio-efd51e149404d77dcc9115e9b25e48535536495dbfe3c0527247bd29ecc48f6a WatchSource:0}: Error finding container efd51e149404d77dcc9115e9b25e48535536495dbfe3c0527247bd29ecc48f6a: Status 404 returned error can't find the container with id efd51e149404d77dcc9115e9b25e48535536495dbfe3c0527247bd29ecc48f6a
Nov 04 01:55:31.640631 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.640572    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:31.670413 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.670385    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jnspf"]
Nov 04 01:55:31.673103 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:55:31.673056    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod534103f4_2d9e_4bd7_ba78_d5e848d3924c.slice/crio-a8c8dea2e42857e417d62279b4afdada39ac9c6b63f44b12722af7f5ca4a2111 WatchSource:0}: Error finding container a8c8dea2e42857e417d62279b4afdada39ac9c6b63f44b12722af7f5ca4a2111: Status 404 returned error can't find the container with id a8c8dea2e42857e417d62279b4afdada39ac9c6b63f44b12722af7f5ca4a2111
Nov 04 01:55:31.684734 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.684700    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqmq6\" (UniqueName: \"kubernetes.io/projected/8369e5ee-521f-4922-9aff-7bcacc063cfa-kube-api-access-jqmq6\") pod \"migrator-66df5d496b-6xdt4\" (UID: \"8369e5ee-521f-4922-9aff-7bcacc063cfa\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4"
Nov 04 01:55:31.696455 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.696282    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4"
Nov 04 01:55:31.854047 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:31.854006    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:32.085527 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.085474    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4"]
Nov 04 01:55:32.090477 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:55:32.090446    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8369e5ee_521f_4922_9aff_7bcacc063cfa.slice/crio-56d9f680e2b84ec9503ce417aba7da15bd784cfabe763862db4173c37e8cebb4 WatchSource:0}: Error finding container 56d9f680e2b84ec9503ce417aba7da15bd784cfabe763862db4173c37e8cebb4: Status 404 returned error can't find the container with id 56d9f680e2b84ec9503ce417aba7da15bd784cfabe763862db4173c37e8cebb4
Nov 04 01:55:32.376827 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.376744    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-lql5t"]
Nov 04 01:55:32.399346 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.399321    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.437766 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.437739    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-lql5t"]
Nov 04 01:55:32.452287 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.452246    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 04 01:55:32.452287 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.452246    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 04 01:55:32.452483 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.452310    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 04 01:55:32.452483 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.452358    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rrqh4\""
Nov 04 01:55:32.452483 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.452417    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 04 01:55:32.536000 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.535968    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:32.536457 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.536040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-key\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.536457 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.536094    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cf6j\" (UniqueName: \"kubernetes.io/projected/53e77f82-4b9d-4e97-9108-3f8f0740925f-kube-api-access-5cf6j\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.536457 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:32.536121    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:32.536457 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:32.536182    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls podName:eb7f58fe-db82-4c9e-bdfd-39725432e053 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:36.536164077 +0000 UTC m=+39.667853838 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls") pod "dns-default-l89qg" (UID: "eb7f58fe-db82-4c9e-bdfd-39725432e053") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:32.536457 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.536124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-cabundle\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.592636 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.592598    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jnspf" event={"ID":"534103f4-2d9e-4bd7-ba78-d5e848d3924c","Type":"ContainerStarted","Data":"a8c8dea2e42857e417d62279b4afdada39ac9c6b63f44b12722af7f5ca4a2111"}
Nov 04 01:55:32.593944 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.593913    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dlxgx" event={"ID":"51998d39-62ca-432f-bac3-3b8142d05e0e","Type":"ContainerStarted","Data":"efd51e149404d77dcc9115e9b25e48535536495dbfe3c0527247bd29ecc48f6a"}
Nov 04 01:55:32.595158 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.595132    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4" event={"ID":"8369e5ee-521f-4922-9aff-7bcacc063cfa","Type":"ContainerStarted","Data":"56d9f680e2b84ec9503ce417aba7da15bd784cfabe763862db4173c37e8cebb4"}
Nov 04 01:55:32.636780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.636479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-key\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.636780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.636574    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cf6j\" (UniqueName: \"kubernetes.io/projected/53e77f82-4b9d-4e97-9108-3f8f0740925f-kube-api-access-5cf6j\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.636780 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.636603    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-cabundle\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.637381 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.637350    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-cabundle\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.637690 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.637666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:32.639894 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.639871    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-key\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.698635 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.698604    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cf6j\" (UniqueName: \"kubernetes.io/projected/53e77f82-4b9d-4e97-9108-3f8f0740925f-kube-api-access-5cf6j\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.719465 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.719265    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:32.839974 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:32.838124    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:33.166196 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:33.165285    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-lql5t"]
Nov 04 01:55:33.177070 ip-10-0-142-191 kubenswrapper[2581]: W1104 01:55:33.177032    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e77f82_4b9d_4e97_9108_3f8f0740925f.slice/crio-a0d45f8ac4f5cd045ae22cfd9aa7551c23f4597559d73253db976b70c3b3a030 WatchSource:0}: Error finding container a0d45f8ac4f5cd045ae22cfd9aa7551c23f4597559d73253db976b70c3b3a030: Status 404 returned error can't find the container with id a0d45f8ac4f5cd045ae22cfd9aa7551c23f4597559d73253db976b70c3b3a030
Nov 04 01:55:33.442910 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:33.442819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:33.442910 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:33.442878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:33.443118 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:33.442975    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:55:33.443118 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:33.442993    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:33.443118 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:33.443035    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:37.443018352 +0000 UTC m=+40.574708128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : configmap "kube-rbac-proxy" not found
Nov 04 01:55:33.443118 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:33.443061    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:37.443043165 +0000 UTC m=+40.574732941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:33.483954 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:33.483887    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:33.601222 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:33.601186    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t" event={"ID":"53e77f82-4b9d-4e97-9108-3f8f0740925f","Type":"ContainerStarted","Data":"a0d45f8ac4f5cd045ae22cfd9aa7551c23f4597559d73253db976b70c3b3a030"}
Nov 04 01:55:33.603425 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:33.603402    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qpbh2" event={"ID":"d91062aa-c9b2-48ac-865a-fc0acb25b4a6","Type":"ContainerStarted","Data":"246e10c4a2298a8cb1de53d6d06c52d652d5e7cd437d5cc48a1088abbd60c081"}
Nov 04 01:55:33.721960 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:33.721901    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qpbh2" podStartSLOduration=2.392293455 podStartE2EDuration="5.721881799s" totalImagesPullingTime="3.329588344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:33.720139901 +0000 UTC m=+36.851829683" watchObservedRunningTime="2026-11-04 01:55:33.721881799 +0000 UTC m=+36.853571583"
Nov 04 01:55:34.611618 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:34.610870    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4" event={"ID":"8369e5ee-521f-4922-9aff-7bcacc063cfa","Type":"ContainerStarted","Data":"e4633849b8bdde7877b8932646b9371041def96181e79461da11f001e59915bd"}
Nov 04 01:55:34.614523 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:34.614473    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lzc9g-h2w8g" event={"ID":"e7d3eb2e-9765-455d-b901-6596bdb56f7e","Type":"ContainerStarted","Data":"7ea8b6a28f05012a07dab156f123273663730cd0c2119c08ab6c2aa30d5bae6a"}
Nov 04 01:55:34.726524 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:34.726446    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4" podStartSLOduration=1.5130950680000002 podStartE2EDuration="3.726426962s" totalImagesPullingTime="2.213331894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:34.726275776 +0000 UTC m=+37.857965559" watchObservedRunningTime="2026-11-04 01:55:34.726426962 +0000 UTC m=+37.858116748"
Nov 04 01:55:34.843153 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:34.843083    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-lzc9g-h2w8g" podStartSLOduration=2.852398484 podStartE2EDuration="5.843065277s" totalImagesPullingTime="2.990666793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:34.842752968 +0000 UTC m=+37.974442745" watchObservedRunningTime="2026-11-04 01:55:34.843065277 +0000 UTC m=+37.974755057"
Nov 04 01:55:36.042653 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:36.042550    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:36.387373 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:36.387285    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:36.569985 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:36.569955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:36.570129 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:36.570100    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:55:36.570189 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:36.570172    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls podName:eb7f58fe-db82-4c9e-bdfd-39725432e053 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:44.570146036 +0000 UTC m=+47.701835801 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls") pod "dns-default-l89qg" (UID: "eb7f58fe-db82-4c9e-bdfd-39725432e053") : secret "dns-default-metrics-tls" not found
Nov 04 01:55:37.478100 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:37.478063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:37.478556 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:37.478113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:37.478556 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:37.478237    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:37.478556 ip-10-0-142-191 kubenswrapper[2581]: E1104 01:55:37.478319    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:45.478299054 +0000 UTC m=+48.609988816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:37.478673 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:37.478580    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:37.623478 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:37.623437    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jnspf" event={"ID":"534103f4-2d9e-4bd7-ba78-d5e848d3924c","Type":"ContainerStarted","Data":"77dc51d09a8d739bb70bb0fe5ee268c91384128d8efa56ecf573566435ddc27c"}
Nov 04 01:55:37.623682 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:37.623611    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:37.624928 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:37.624907    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t" event={"ID":"53e77f82-4b9d-4e97-9108-3f8f0740925f","Type":"ContainerStarted","Data":"1a2852f3b7654e4c5a69d3d72cbd77f6378b9c3d448ef8cad5e3f9f6dbe3aad2"}
Nov 04 01:55:37.721086 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:37.721060    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tm85n_dff859f4-bd6d-40b5-8a3d-fccf7e0baf67/dns-node-resolver/0.log"
Nov 04 01:55:37.739340 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:37.739297    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jnspf" podStartSLOduration=34.712476123 podStartE2EDuration="39.739284095s" totalImagesPullingTime="5.026807972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:37.738674991 +0000 UTC m=+40.870364776" watchObservedRunningTime="2026-11-04 01:55:37.739284095 +0000 UTC m=+40.870973877"
Nov 04 01:55:37.854020 ip-10-0-142-191 kubenswrapper[2581]: I1104 01:55:37.853976    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t" podStartSLOduration=2.316183383 podStartE2EDuration="5.853964345s" totalImagesPullingTime="3.537780962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:37.853664579 +0000 UTC m=+40.985354362" watchObservedRunningTime="2026-11-04 01:55:37.853964345 +0000 UTC m=+40.985654127"
Nov 04 01:55:38.144384 ip-10-0-142-191 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 01:55:38.153056 ip-10-0-142-191 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 01:55:38.153223 ip-10-0-142-191 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 01:55:38.153275 ip-10-0-142-191 systemd[1]: kubelet.service: Consumed 2.518s CPU time.
Nov 04 01:55:38.164161 ip-10-0-142-191 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 01:55:38.222731 ip-10-0-142-191 kubenswrapper[4512]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:55:38.222731 ip-10-0-142-191 kubenswrapper[4512]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 01:55:38.222731 ip-10-0-142-191 kubenswrapper[4512]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:55:38.222731 ip-10-0-142-191 kubenswrapper[4512]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:55:38.224678 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224603    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:38.224678 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224677    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224682    4512 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224685    4512 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224688    4512 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224691    4512 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224694    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224696    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224699    4512 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224701    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224704    4512 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224706    4512 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224708    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224711    4512 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224713    4512 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224716    4512 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224718    4512 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224720    4512 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224723    4512 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224725    4512 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224728    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:38.224747 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224730    4512 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224732    4512 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224735    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224743    4512 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224746    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224749    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224752    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224754    4512 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224757    4512 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224760    4512 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224762    4512 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224765    4512 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224767    4512 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224770    4512 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224772    4512 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224775    4512 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224777    4512 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224780    4512 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224782    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224785    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:38.225229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224787    4512 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224790    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224792    4512 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224795    4512 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224807    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224810    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224812    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224815    4512 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224817    4512 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224820    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224822    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224825    4512 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224827    4512 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224830    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224833    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224836    4512 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224838    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224841    4512 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224844    4512 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224846    4512 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:38.225734 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224849    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224851    4512 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224854    4512 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224856    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224859    4512 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224862    4512 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224864    4512 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224867    4512 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224871    4512 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224875    4512 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224877    4512 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224880    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224882    4512 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224885    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224893    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224895    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224898    4512 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224901    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224903    4512 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:38.226237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224905    4512 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224908    4512 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224910    4512 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224912    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224915    4512 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224918    4512 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224920    4512 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224923    4512 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224925    4512 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224928    4512 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224931    4512 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224933    4512 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224936    4512 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224938    4512 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224940    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224943    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224945    4512 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224948    4512 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224950    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224952    4512 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:38.226729 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224955    4512 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224959    4512 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224962    4512 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224965    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224967    4512 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.224970    4512 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225768    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225777    4512 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225781    4512 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225786    4512 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225790    4512 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225793    4512 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225796    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225800    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225803    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225805    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225809    4512 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225813    4512 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:38.227210 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225817    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225820    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225823    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225826    4512 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225829    4512 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225832    4512 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225835    4512 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225837    4512 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225840    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225842    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225845    4512 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225847    4512 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225850    4512 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225853    4512 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225855    4512 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225857    4512 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225860    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225862    4512 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225864    4512 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:38.227663 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225867    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225869    4512 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225871    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225874    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225877    4512 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225879    4512 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225882    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225884    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225887    4512 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225889    4512 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225892    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225894    4512 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225897    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225899    4512 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225902    4512 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225904    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225906    4512 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225909    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225911    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225913    4512 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:38.228106 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225916    4512 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225918    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225921    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225923    4512 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225925    4512 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225928    4512 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225930    4512 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225932    4512 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225935    4512 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225937    4512 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225940    4512 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225942    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225945    4512 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225947    4512 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225949    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225952    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225955    4512 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225958    4512 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225960    4512 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225963    4512 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:38.228630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225965    4512 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225967    4512 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225970    4512 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225972    4512 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225975    4512 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225980    4512 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225982    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225985    4512 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225987    4512 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225989    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225992    4512 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225994    4512 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225997    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.225999    4512 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226002    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226004    4512 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226007    4512 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226009    4512 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226012    4512 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226014    4512 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:38.229171 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226016    4512 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226020    4512 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226022    4512 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226025    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226027    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226030    4512 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226032    4512 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226035    4512 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226037    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226039    4512 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226042    4512 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226045    4512 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226047    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226050    4512 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226053    4512 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226709    4512 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226717    4512 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226722    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226724    4512 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:38.229717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226727    4512 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226730    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226733    4512 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226735    4512 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226738    4512 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226741    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226744    4512 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226746    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226750    4512 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226754    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226757    4512 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226760    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226762    4512 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226766    4512 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226768    4512 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226771    4512 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226773    4512 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226776    4512 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226779    4512 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:38.230232 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226781    4512 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226784    4512 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226786    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226789    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226792    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226794    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226798    4512 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226802    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226804    4512 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226807    4512 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226810    4512 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226813    4512 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226816    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226818    4512 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226820    4512 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226823    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226826    4512 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226828    4512 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226831    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:38.230709 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226833    4512 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226836    4512 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226838    4512 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226841    4512 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226843    4512 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226845    4512 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226848    4512 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226850    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226853    4512 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226855    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226858    4512 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226860    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226864    4512 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226866    4512 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226869    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226871    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226873    4512 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226876    4512 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226879    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226882    4512 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:38.231164 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226884    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226887    4512 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226889    4512 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226892    4512 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226896    4512 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226898    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226901    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226903    4512 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226906    4512 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226908    4512 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226910    4512 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226912    4512 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226915    4512 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226917    4512 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226920    4512 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226922    4512 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226925    4512 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226927    4512 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226929    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:38.231778 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226932    4512 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226934    4512 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226936    4512 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226939    4512 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226941    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226944    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226946    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226949    4512 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226951    4512 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226953    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226956    4512 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226958    4512 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226961    4512 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226963    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226966    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226968    4512 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226971    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226975    4512 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226978    4512 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:38.232234 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226980    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226983    4512 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226985    4512 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226988    4512 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226991    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.226993    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227078    4512 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227085    4512 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227094    4512 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227099    4512 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227104    4512 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227108    4512 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227112    4512 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227116    4512 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227119    4512 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227122    4512 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227125    4512 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227128    4512 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227131    4512 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227134    4512 flags.go:64] FLAG: --cgroup-root=""
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227137    4512 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227140    4512 flags.go:64] FLAG: --client-ca-file=""
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227142    4512 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227145    4512 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 01:55:38.232694 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227148    4512 flags.go:64] FLAG: --cluster-domain=""
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227151    4512 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227154    4512 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227158    4512 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227161    4512 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227165    4512 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227169    4512 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227172    4512 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227176    4512 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227178    4512 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227181    4512 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227184    4512 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227187    4512 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227189    4512 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227196    4512 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227199    4512 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227201    4512 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227204    4512 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227208    4512 flags.go:64] FLAG: --enable-server="true"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227211    4512 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227215    4512 flags.go:64] FLAG: --event-burst="100"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227219    4512 flags.go:64] FLAG: --event-qps="50"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227222    4512 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227225    4512 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 01:55:38.233292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227228    4512 flags.go:64] FLAG: --eviction-hard=""
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227232    4512 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227234    4512 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227237    4512 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227240    4512 flags.go:64] FLAG: --eviction-soft=""
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227243    4512 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227246    4512 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227249    4512 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227252    4512 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227254    4512 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227257    4512 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227261    4512 flags.go:64] FLAG: --feature-gates=""
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227265    4512 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227268    4512 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227271    4512 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227274    4512 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227277    4512 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227280    4512 flags.go:64] FLAG: --help="false"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227283    4512 flags.go:64] FLAG: --hostname-override="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227286    4512 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227289    4512 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227291    4512 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227294    4512 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 01:55:38.234285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227297    4512 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227300    4512 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227303    4512 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227306    4512 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227309    4512 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227312    4512 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227315    4512 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227318    4512 flags.go:64] FLAG: --kube-reserved=""
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227320    4512 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227323    4512 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227326    4512 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227329    4512 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227331    4512 flags.go:64] FLAG: --lock-file=""
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227334    4512 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227337    4512 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227340    4512 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227345    4512 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227347    4512 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227350    4512 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227353    4512 flags.go:64] FLAG: --logging-format="text"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227356    4512 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227359    4512 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227362    4512 flags.go:64] FLAG: --manifest-url=""
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227365    4512 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227370    4512 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 01:55:38.234872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227373    4512 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227376    4512 flags.go:64] FLAG: --max-pods="110"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227379    4512 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227382    4512 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227385    4512 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227388    4512 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227390    4512 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227393    4512 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227396    4512 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227402    4512 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227407    4512 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227410    4512 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227413    4512 flags.go:64] FLAG: --pod-cidr=""
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227415    4512 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227418    4512 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227421    4512 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227423    4512 flags.go:64] FLAG: --port="10250"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227426    4512 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227429    4512 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0cc1e9cb2e0ca73a7"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227432    4512 flags.go:64] FLAG: --qos-reserved=""
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227435    4512 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227438    4512 flags.go:64] FLAG: --register-node="true"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227440    4512 flags.go:64] FLAG: --register-with-taints=""
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227444    4512 flags.go:64] FLAG: --registry-burst="10"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227447    4512 flags.go:64] FLAG: --registry-qps="5"
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227450    4512 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 01:55:38.235469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227453    4512 flags.go:64] FLAG: --reserved-memory=""
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227456    4512 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227459    4512 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227463    4512 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227466    4512 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227468    4512 flags.go:64] FLAG: --runonce="false"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227471    4512 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227474    4512 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227476    4512 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227479    4512 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227482    4512 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227485    4512 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227488    4512 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227490    4512 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227493    4512 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227496    4512 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227500    4512 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227517    4512 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227521    4512 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227523    4512 flags.go:64] FLAG: --system-cgroups=""
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227526    4512 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227531    4512 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227534    4512 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227537    4512 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227540    4512 flags.go:64] FLAG: --tls-min-version=""
Nov 04 01:55:38.236106 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227542    4512 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227545    4512 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227548    4512 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227551    4512 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227553    4512 flags.go:64] FLAG: --v="2"
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227557    4512 flags.go:64] FLAG: --version="false"
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227561    4512 flags.go:64] FLAG: --vmodule=""
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227565    4512 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.227568    4512 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227713    4512 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227716    4512 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227720    4512 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227723    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227725    4512 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227728    4512 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227730    4512 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227733    4512 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227735    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227737    4512 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227740    4512 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227742    4512 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227745    4512 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:38.236782 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227747    4512 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227751    4512 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227753    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227756    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227758    4512 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227761    4512 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227763    4512 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227766    4512 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227768    4512 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227770    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227773    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227775    4512 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227777    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227780    4512 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227782    4512 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227785    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227787    4512 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227790    4512 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227793    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:38.237321 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227796    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227800    4512 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227803    4512 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227806    4512 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227809    4512 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227811    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227814    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227816    4512 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227819    4512 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227821    4512 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227823    4512 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227826    4512 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227828    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227831    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227834    4512 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227837    4512 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227839    4512 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227842    4512 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227844    4512 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:38.237792 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227847    4512 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227849    4512 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227852    4512 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227854    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227856    4512 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227859    4512 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227861    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227863    4512 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227866    4512 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227868    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227871    4512 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227874    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227877    4512 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227879    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227881    4512 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227884    4512 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227886    4512 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227889    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227891    4512 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227894    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:38.238237 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227897    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227901    4512 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227904    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227907    4512 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227911    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227913    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227916    4512 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227919    4512 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227922    4512 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227924    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227927    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227929    4512 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227931    4512 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227934    4512 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227936    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227939    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227941    4512 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227943    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227946    4512 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:38.238807 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227948    4512 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227951    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227953    4512 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227955    4512 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227959    4512 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227962    4512 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227964    4512 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227966    4512 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227969    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227972    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227974    4512 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227977    4512 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227979    4512 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227982    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227984    4512 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.227987    4512 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:38.239300 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.228011    4512 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228045    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228049    4512 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228052    4512 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228056    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228059    4512 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228062    4512 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228065    4512 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228067    4512 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228070    4512 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228072    4512 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228075    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228078    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228080    4512 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228082    4512 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228085    4512 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228087    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228089    4512 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228092    4512 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:38.239984 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228094    4512 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228096    4512 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228100    4512 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228102    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228105    4512 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228107    4512 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228110    4512 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228113    4512 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228115    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228118    4512 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228121    4512 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228123    4512 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228125    4512 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228128    4512 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228130    4512 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228133    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228135    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228139    4512 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228142    4512 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228144    4512 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:38.240736 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228147    4512 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228149    4512 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228152    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228210    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228213    4512 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228216    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228218    4512 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228221    4512 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228224    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228226    4512 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228228    4512 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228231    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228233    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228237    4512 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228242    4512 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228245    4512 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228247    4512 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228250    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228252    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:38.241437 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228255    4512 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228257    4512 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228260    4512 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228263    4512 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228265    4512 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228268    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228270    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228273    4512 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228275    4512 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228278    4512 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228282    4512 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228284    4512 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228287    4512 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228289    4512 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228291    4512 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228294    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228296    4512 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228298    4512 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228301    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:38.242129 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228303    4512 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228306    4512 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228308    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228310    4512 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228313    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228315    4512 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228317    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228320    4512 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228323    4512 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228326    4512 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228328    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228331    4512 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228333    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228335    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228339    4512 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228343    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228346    4512 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228349    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228352    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228354    4512 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:38.243160 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228356    4512 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228359    4512 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228361    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228365    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228367    4512 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228370    4512 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228373    4512 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228375    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228378    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.228380    4512 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.232591    4512 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.232607    4512 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.232625    4512 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.232629    4512 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232697    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232703    4512 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232706    4512 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232709    4512 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232712    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232715    4512 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:38.244169 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232718    4512 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232721    4512 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232723    4512 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232726    4512 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232729    4512 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232732    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232734    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232737    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232740    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232743    4512 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232746    4512 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232748    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232751    4512 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232753    4512 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232756    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232759    4512 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232761    4512 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232763    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232766    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:38.247082 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232768    4512 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232776    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232779    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232782    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232784    4512 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232787    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232789    4512 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232791    4512 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232794    4512 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232796    4512 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232799    4512 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232801    4512 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232804    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232806    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232809    4512 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232818    4512 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232821    4512 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232823    4512 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232826    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232828    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:38.247630 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232831    4512 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232835    4512 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232839    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232842    4512 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232845    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232848    4512 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232850    4512 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232853    4512 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232856    4512 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232860    4512 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232863    4512 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232866    4512 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232868    4512 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232871    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232879    4512 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232882    4512 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232884    4512 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232887    4512 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232889    4512 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:38.248112 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232892    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232894    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232897    4512 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232899    4512 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232901    4512 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232904    4512 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232907    4512 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232909    4512 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232912    4512 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232914    4512 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232916    4512 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232919    4512 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232921    4512 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232924    4512 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232926    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232929    4512 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232931    4512 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232933    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232936    4512 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232938    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:38.248575 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232940    4512 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232943    4512 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232945    4512 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232947    4512 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232950    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232952    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232954    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232957    4512 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232964    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232967    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232969    4512 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232972    4512 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232974    4512 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232976    4512 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232979    4512 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232981    4512 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232984    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232986    4512 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232988    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232991    4512 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:38.249066 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232993    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.232995    4512 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.233022    4512 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233356    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233372    4512 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233381    4512 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233385    4512 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233399    4512 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233406    4512 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233410    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233415    4512 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233422    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233426    4512 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233430    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233434    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233439    4512 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:38.249557 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233442    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233446    4512 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233452    4512 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233456    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233465    4512 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233521    4512 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233527    4512 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233531    4512 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233535    4512 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233539    4512 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233543    4512 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233547    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233551    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233555    4512 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233559    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233563    4512 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233572    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233576    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233579    4512 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:38.249946 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233584    4512 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233588    4512 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233592    4512 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233597    4512 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233600    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233604    4512 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233608    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233612    4512 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233616    4512 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233620    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233629    4512 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233632    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233636    4512 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233640    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233644    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233648    4512 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233654    4512 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233662    4512 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233667    4512 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:38.250434 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233702    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233707    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233711    4512 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233720    4512 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233724    4512 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233728    4512 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233732    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233736    4512 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233740    4512 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233745    4512 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233749    4512 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233753    4512 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233757    4512 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233762    4512 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233766    4512 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233775    4512 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233781    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233786    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233789    4512 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233794    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:38.250899 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233798    4512 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233802    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233806    4512 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233810    4512 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233816    4512 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233822    4512 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233826    4512 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233831    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233840    4512 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233845    4512 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233849    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233853    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233857    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233892    4512 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233896    4512 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233901    4512 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233904    4512 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233909    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233912    4512 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:38.251378 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233916    4512 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233926    4512 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233931    4512 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233935    4512 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233939    4512 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233943    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233947    4512 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233951    4512 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233955    4512 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233961    4512 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233965    4512 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233969    4512 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233973    4512 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233977    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233986    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.233990    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:38.251878 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.234039    4512 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234261    4512 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234276    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234279    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234282    4512 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234288    4512 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234293    4512 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234298    4512 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234303    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234307    4512 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234311    4512 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234321    4512 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234325    4512 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234331    4512 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234334    4512 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234337    4512 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234343    4512 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234436    4512 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234441    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234443    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:55:38.252292 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234446    4512 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234449    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234452    4512 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234454    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234457    4512 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234460    4512 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234465    4512 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234470    4512 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234473    4512 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234476    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234478    4512 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234481    4512 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234484    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234486    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234489    4512 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234493    4512 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234496    4512 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234499    4512 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234501    4512 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:55:38.252972 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234522    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234526    4512 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234529    4512 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234532    4512 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234534    4512 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234538    4512 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234540    4512 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234543    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234546    4512 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234549    4512 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234551    4512 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234554    4512 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234556    4512 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234559    4512 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234561    4512 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234566    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234569    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234572    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234574    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234577    4512 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:55:38.253744 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234579    4512 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234582    4512 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234584    4512 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234587    4512 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234589    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234592    4512 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234594    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234597    4512 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234599    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234601    4512 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234604    4512 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234606    4512 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234609    4512 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234611    4512 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234613    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234616    4512 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234618    4512 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234621    4512 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234623    4512 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234625    4512 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:55:38.254497 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234628    4512 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234631    4512 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234634    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234636    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234639    4512 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234641    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234644    4512 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234646    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234649    4512 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234651    4512 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234654    4512 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234656    4512 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234659    4512 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234661    4512 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234664    4512 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234667    4512 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234669    4512 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234672    4512 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234674    4512 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:55:38.255343 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234677    4512 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234679    4512 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234681    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234684    4512 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234686    4512 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234689    4512 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234691    4512 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234696    4512 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:38.234700    4512 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.234791    4512 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.235751    4512 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.235804    4512 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.235987    4512 server.go:1014] "Starting client certificate rotation"
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.236122    4512 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.236181    4512 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 01:49:58 +0000 UTC" deadline="2028-06-02 08:43:09.168464395 +0000 UTC"
Nov 04 01:55:38.256057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.236216    4512 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13830h47m30.932250373s"
Nov 04 01:55:38.256648 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.236472    4512 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:55:38.256648 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.237544    4512 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:55:38.256648 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.239165    4512 log.go:25] "Validated CRI v1 runtime API"
Nov 04 01:55:38.256648 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.241089    4512 log.go:25] "Validated CRI v1 image API"
Nov 04 01:55:38.256648 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.242254    4512 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 01:55:38.256648 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.244036    4512 fs.go:135] Filesystem UUIDs: map[4de28615-663a-4331-b88e-e7c5b8c95908:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 fd248c9a-c51a-405e-963f-90e25cf0a42c:/dev/nvme0n1p3]
Nov 04 01:55:38.256918 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.244056    4512 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0933120d5faf936a058dde92a7f114405b149969a72d2801c2b6e853f9d4260d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0933120d5faf936a058dde92a7f114405b149969a72d2801c2b6e853f9d4260d/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/176cec00bd80a2d82a631b074d61bfed927d9dea6c40bb261c9653dea5e1b48c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/176cec00bd80a2d82a631b074d61bfed927d9dea6c40bb261c9653dea5e1b48c/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/56d9f680e2b84ec9503ce417aba7da15bd784cfabe763862db4173c37e8cebb4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/56d9f680e2b84ec9503ce417aba7da15bd784cfabe763862db4173c37e8cebb4/userdata/shm major:0 minor:223 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/66634df91a7cd64cbc53723f4403b81d22922a9c62869f8cb686560f3a92f823/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/66634df91a7cd64cbc53723f4403b81d22922a9c62869f8cb686560f3a92f823/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9961b1778e0f5bc3f822cd88426734388acf849099e6fa143c63c8a29743d13e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9961b1778e0f5bc3f822cd88426734388acf849099e6fa143c63c8a29743d13e/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a0d45f8ac4f5cd045ae22cfd9aa7551c23f4597559d73253db976b70c3b3a030/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a0d45f8ac4f5cd045ae22cfd9aa7551c23f4597559d73253db976b70c3b3a030/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a6dc01b4f280c69b31a786329af3165111eb7fbc4cbe6fc98d352341a87ad56c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a6dc01b4f280c69b31a786329af3165111eb7fbc4cbe6fc98d352341a87ad56c/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a700b7f4b57af9ece70802725c34dec353f916ec71c513e5d3d68d54c4ed7f3e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a700b7f4b57af9ece70802725c34dec353f916ec71c513e5d3d68d54c4ed7f3e/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a8c8dea2e42857e417d62279b4afdada39ac9c6b63f44b12722af7f5ca4a2111/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a8c8dea2e42857e417d62279b4afdada39ac9c6b63f44b12722af7f5ca4a2111/userdata/shm major:0 minor:216 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b3eb0ec95449a0a40f3c1fb1812e3cec836189a48d2c75da55cf0a38c27799ee/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b3eb0ec95449a0a40f3c1fb1812e3cec836189a48d2c75da55cf0a38c27799ee/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b7fb3b328f8cc69b175873fe57227ba00e39c5a57be7129c7ad67de8d4418273/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b7fb3b328f8cc69b175873fe57227ba00e39c5a57be7129c7ad67de8d4418273/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dc2a797ea8cd7857e843af4d0308e69f20e8de727faa0c34048bfab6331b9160/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dc2a797ea8cd7857e843af4d0308e69f20e8de727faa0c34048bfab6331b9160/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e14b06a2ee640758ad0d97a0eac3923c7a94e83289be79f1a5934374965e31ca/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e14b06a2ee640758ad0d97a0eac3923c7a94e83289be79f1a5934374965e31ca/userdata/shm major:0 minor:85 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e3fcec2a2abff87025a81d319b1a140f47f7b17c6ad193209468b49363536050/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e3fcec2a2abff87025a81d319b1a140f47f7b17c6ad193209468b49363536050/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ec2580eb977c12486b4e4f186385502f4763f9e61963fb08996c490edba872bf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ec2580eb977c12486b4e4f186385502f4763f9e61963fb08996c490edba872bf/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/efd51e149404d77dcc9115e9b25e48535536495dbfe3c0527247bd29ecc48f6a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/efd51e149404d77dcc9115e9b25e48535536495dbfe3c0527247bd29ecc48f6a/userdata/shm major:0 minor:214 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ca946c4-34f5-4443-9ecf-e634055dacc0/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/1ca946c4-34f5-4443-9ecf-e634055dacc0/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ca946c4-34f5-4443-9ecf-e634055dacc0/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/1ca946c4-34f5-4443-9ecf-e634055dacc0/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ca946c4-34f5-4443-9ecf-e634055dacc0/volumes/kubernetes.io~projected/kube-api-access-mcrk5:{mountpoint:/var/lib/kubelet/pods/1ca946c4-34f5-4443-9ecf-e634055dacc0/volumes/kubernetes.io~projected/kube-api-access-mcrk5 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30ad9387-03e3-49d0-8ff8-b4959c6f343f/volumes/kubernetes.io~projected/kube-api-access-m8k7f:{mountpoint:/var/lib/kubelet/pods/30ad9387-03e3-49d0-8ff8-b4959c6f343f/volumes/kubernetes.io~projected/kube-api-access-m8k7f major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51998d39-62ca-432f-bac3-3b8142d05e0e/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/51998d39-62ca-432f-bac3-3b8142d05e0e/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/534103f4-2d9e-4bd7-ba78-d5e848d3924c/volumes/kubernetes.io~projected/kube-api-access-ptdph:{mountpoint:/var/lib/kubelet/pods/534103f4-2d9e-4bd7-ba78-d5e848d3924c/volumes/kubernetes.io~projected/kube-api-access-ptdph major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53e77f82-4b9d-4e97-9108-3f8f0740925f/volumes/kubernetes.io~projected/kube-api-access-5cf6j:{mountpoint:/var/lib/kubelet/pods/53e77f82-4b9d-4e97-9108-3f8f0740925f/volumes/kubernetes.io~projected/kube-api-access-5cf6j major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53e77f82-4b9d-4e97-9108-3f8f0740925f/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/53e77f82-4b9d-4e97-9108-3f8f0740925f/volumes/kubernetes.io~secret/signing-key major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8369e5ee-521f-4922-9aff-7bcacc063cfa/volumes/kubernetes.io~projected/kube-api-access-jqmq6:{mountpoint:/var/lib/kubelet/pods/8369e5ee-521f-4922-9aff-7bcacc063cfa/volumes/kubernetes.io~projected/kube-api-access-jqmq6 major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87120866-99b9-46d6-9883-6244b38c38ae/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/87120866-99b9-46d6-9883-6244b38c38ae/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87120866-99b9-46d6-9883-6244b38c38ae/volumes/kubernetes.io~projected/kube-api-access-kchqw:{mountpoint:/var/lib/kubelet/pods/87120866-99b9-46d6-9883-6244b38c38ae/volumes/kubernetes.io~projected/kube-api-access-kchqw major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87120866-99b9-46d6-9883-6244b38c38ae/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/87120866-99b9-46d6-9883-6244b38c38ae/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aad0ba21-8234-4c49-9c72-183b7b1dbcc5/volumes/kubernetes.io~projected/kube-api-access-gjg5x:{mountpoint:/var/lib/kubelet/pods/aad0ba21-8234-4c49-9c72-183b7b1dbcc5/volumes/kubernetes.io~projected/kube-api-access-gjg5x major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b94155ab-70f0-4efa-a01c-7303db0d8cb5/volumes/kubernetes.io~projected/kube-api-access-zcwq8:{mountpoint:/var/lib/kubelet/pods/b94155ab-70f0-4efa-a01c-7303db0d8cb5/volumes/kubernetes.io~projected/kube-api-access-zcwq8 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdf00c2a-249a-4bc8-af8d-89d2f080eb16/volumes/kubernetes.io~projected/kube-api-access-g8g55:{mountpoint:/var/lib/kubelet/pods/bdf00c2a-249a-4bc8-af8d-89d2f080eb16/volumes/kubernetes.io~projected/kube-api-access-g8g55 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c54f5f65-a78e-4dd3-908f-7242d386dd33/volumes/kubernetes.io~projected/kube-api-access-xdrl6:{mountpoint:/var/lib/kubelet/pods/c54f5f65-a78e-4dd3-908f-7242d386dd33/volumes/kubernetes.io~projected/kube-api-access-xdrl6 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d91062aa-c9b2-48ac-865a-fc0acb25b4a6/volumes/kubernetes.io~projected/kube-api-access-vl6p9:{mountpoint:/var/lib/kubelet/pods/d91062aa-c9b2-48ac-865a-fc0acb25b4a6/volumes/kubernetes.io~projected/kube-api-access-vl6p9 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d91062aa-c9b2-48ac-865a-fc0acb25b4a6/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/d91062aa-c9b2-48ac-865a-fc0acb25b4a6/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dfca312e-b151-4c3a-99bd-dd820925264b/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/dfca312e-b151-4c3a-99bd-dd820925264b/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67/volumes/kubernetes.io~projected/kube-api-access-px9wr:{mountpoint:/var/lib/kubelet/pods/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67/volumes/kubernetes.io~projected/kube-api-access-px9wr major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e7d3eb2e-9765-455d-b901-6596bdb56f7e/volumes/kubernetes.io~projected/kube-api-access-r9k72:{mountpoint:/var/lib/kubelet/pods/e7d3eb2e-9765-455d-b901-6596bdb56f7e/volumes/kubernetes.io~projected/kube-api-access-r9k72 major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e882a774-f295-43dd-916c-e20c2df7f05a/volumes/kubernetes.io~projected/kube-api-access-4zzbc:{mountpoint:/var/lib/kubelet/pods/e882a774-f295-43dd-916c-e20c2df7f05a/volumes/kubernetes.io~projected/kube-api-access-4zzbc major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb7f58fe-db82-4c9e-bdfd-39725432e053/volumes/kubernetes.io~projected/kube-api-access-hjzd4:{mountpoint:/var/lib/kubelet/pods/eb7f58fe-db82-4c9e-bdfd-39725432e053/volumes/kubernetes.io~projected/kube-api-access-hjzd4 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4fe5f4e-25f4-400c-9efb-f2f98757c74e/volumes/kubernetes.io~projected/kube-api-access-7xjbg:{mountpoint:/var/lib/kubelet/pods/f4fe5f4e-25f4-400c-9efb-f2f98757c74e/volumes/kubernetes.io~projected/kube-api-access-7xjbg major:0 minor:62 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/468f4ebab287958d90d9f0db0c8291f5bf2d263aeeddededf61af494cbc95777/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/6e4630e08f042ce6d994f4a6eeedafc3271320f90ad68d26cd75fcfd152e613a/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/4604dc585b61b5fed27250ddfedb387c7ae3ee672f9e457b88dffc80e38a65c3/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/6de3bdf1c87d117cc1fe8f18e0c3969ec0f025cbd1e8dadcb468722b1f10d1ff/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/1fe3106ff2e4751b0169eec82997142a5b687d45bad90d3b4e768cac3757d8d1/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/676ebeb13149abc17f09544362aad7fe50233f0d87a012645cf1606c721f9fca/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/abca56f6d8120c46611ac4abff3c1609274c80b24f6ce838bd9dcd1c8b63f05b/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/1c42616beb91ce036dfbdbfe9f060a2c02bf70a94725a34d5f8481f4e55eebf2/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/99b7a4f6ef63e4f8d10fd8a33627e637d6cbd8e61870ed6874b79854952f7fda/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/3fdeff30126d04d6ccf81cf64280b76e15e8b4b4cdac8f259658f839af3a114a/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-141:{mountpoint:/var/lib/containers/storage/overlay/92e0e7ba6235b97411772f68c1d0b971df99d6dd5f7388e18097dc21956afd6b/merged major:0 minor:141 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/126949c4b1bb47264f9b318e7dbe4aaa4efddda405130435099e0a41f293cac4/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/538b54f6f32a84bf7c5be8d2c75eded930be137507b3bfe199a40c7f8c45dcf6/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/9f365addc4520da4e593368ea0d44fa1f317765624487f44f8e495116a1d6363/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/1ce3046baaca1c17d4938665dbcbe89c1e52719f07a0d3af16d4d881f855b301/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/716f506c0044d722ace95fc4eb095a024c012fc5d82b89f73d78ad5843ffdaba/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-171:{mountpoint:/var/lib/containers/storage/overlay/87e4b17524a110aeeafec564e8cb8e3b59a8629b08d67c4c087e0bbf501efc55/merged major:0 minor:171 fsType:overlay blockSize:0} overlay_0-174:{mountpoint:/var/lib/containers/storage/overlay/18bc1b7d331e6ea4a9b7056234aac6a4579f7178989be9ee49693dd98a6c19e2/merged major:0 minor:174 fsType:overlay blockSize:0} overlay_0-176:{mountpoint:/var/lib/containers/storage/overlay/7b6c3f394f143957fd4fd8f8eb17a771f2005bd0b9da902e8c202c7f80afcc56/merged major:0 minor:176 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/af2eb3f7de1bc00d4da599bd52179bb74b5b6ad74867071de8ed37cfa49c2993/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/6685007183b5bc0b7e523dfc3cb88de36cc1ef1eec5a598088776bf1f06e2768/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/a11b332a07bf56394b5043432846fd7414fad62adbc5f5d80b0c2c60f546a1b1/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-210:{mountpoint:/var/lib/containers/storage/overlay/ab839f994223e16e50744f6ffb429e1b3c21dbb2f4b81295fbeb1054fdf42165/merged major:0 minor:210 fsType:overlay blockSize:0} overlay_0-218:{mountpoint:/var/lib/containers/storage/overlay/17dd5b355d65b6131217b91a0ecc6e5757bbf86d98e88c2060f7f8aef595f383/merged major:0 minor:218 fsType:overlay blockSize:0} overlay_0-220:{mountpoint:/var/lib/containers/storage/overlay/91723871400f52516f5bf76cdaa90e6719e8c2ab7a124e8b8887409679367320/merged major:0 minor:220 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/e3120171e163e37a5a0e3df88eb1b163c337c606dbf618e819a4419a9150236f/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-231:{mountpoint:/var/lib/containers/storage/overlay/8a25d42f0b9d945b3b7296d8da40442eaef5d7cf5b409a63c2b937a607f89f0b/merged major:0 minor:231 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/a9ae05ecf57219fa7436dc55b8381d142483fe4aae21527743a573f0e4048c55/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/285537570f599a8b0bde02ed7bde2f7bf7ab81929ec9a4068698069418608d6c/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/819695f5ddc7796679ba2b565e567de201833a50cf5437fc048b245733dbd203/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/4a5a9261e41138a4577ed7bb7b57200fddd8beb221440d0adc8ef10c2a3161af/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/6ccd1d5d5ec96bb096db2736820634e083801bd846bca19f7938415625f4471a/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/2e25953765c487ce18888acce06e7566ddc0f1f0437ef875b7710f0a33ec73ab/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/94bdfc2949f14554033dad12abf4de3af1fee4ea34bf3011630cd164131409f8/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/280c3048086bd74e122e1b118fa744700d07f4384e4670e8e0ed30797bedd298/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/0a98b829821832da57c21a143a6720f5d9df3e68a2bc2b4e253247e5883fe36d/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/fd84ca0085d5611f7d01ac351e728f0b8195fde2020c8c006799077309758b8c/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/b2753903b78f2076c505be588dc29685e5ca968058488fbbec01054648d6b85a/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/e12c157df77428b5d03f49821ce0bfa54d08c93b3acdf267f50bfdc34d96e178/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-87:{mountpoint:/var/lib/containers/storage/overlay/d8d8fd52920827a58a54b314f30271b583dd3fd509d6e6dbc8e10ed34db01e4a/merged major:0 minor:87 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/05332e8724c6bd55c4c943fd2f8b9f9d809f8ae0f6c5f89e8f3e3c1fbf578b0a/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/ec1f155adfb7c88f7c55a6af612e7cb5af2f6331286ed292a6f2e5c1336caf8a/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/d2ab1924b387da8bd5c4d244a50a2673ba071ab3640ecbfc216f649bc7732ae9/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.250990    4512 manager.go:217] Machine: {Timestamp:2026-11-04 01:55:38.250524987 +0000 UTC m=+0.055981608 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200499 MemoryCapacity:32812023808 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec254f6ad69576a68578c0ab18e419a3 SystemUUID:ec254f6a-d695-76a6-8578-c0ab18e419a3 BootID:bd868ee2-6be4-45ed-abf1-dbd4a6c45073 Filesystems:[{Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/87120866-99b9-46d6-9883-6244b38c38ae/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/eb7f58fe-db82-4c9e-bdfd-39725432e053/volumes/kubernetes.io~projected/kube-api-access-hjzd4 DeviceMajor:0 DeviceMinor:201 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a8c8dea2e42857e417d62279b4afdada39ac9c6b63f44b12722af7f5ca4a2111/userdata/shm DeviceMajor:0 DeviceMinor:216 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/176cec00bd80a2d82a631b074d61bfed927d9dea6c40bb261c9653dea5e1b48c/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-176 DeviceMajor:0 DeviceMinor:176 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a700b7f4b57af9ece70802725c34dec353f916ec71c513e5d3d68d54c4ed7f3e/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-174 DeviceMajor:0 DeviceMinor:174 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/53e77f82-4b9d-4e97-9108-3f8f0740925f/volumes/kubernetes.io~projected/kube-api-access-5cf6j DeviceMajor:0 DeviceMinor:228 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e3fcec2a2abff87025a81d319b1a140f47f7b17c6ad193209468b49363536050/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-210 DeviceMajor:0 DeviceMinor:210 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-218 DeviceMajor:0 DeviceMinor:218 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b3eb0ec95449a0a40f3c1fb1812e3cec836189a48d2c75da55cf0a38c27799ee/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ec2580eb977c12486b4e4f186385502f4763f9e61963fb08996c490edba872bf/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b7fb3b328f8cc69b175873fe57227ba00e39c5a57be7129c7ad67de8d4418273/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/e7d3eb2e-9765-455d-b901-6596bdb56f7e/volumes/kubernetes.io~projected/kube-api-access-r9k72 DeviceMajor:0 DeviceMinor:207 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/87120866-99b9-46d6-9883-6244b38c38ae/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/c54f5f65-a78e-4dd3-908f-7242d386dd33/volumes/kubernetes.io~projected/kube-api-access-xdrl6 DeviceMajor:0 DeviceMinor:66 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a6dc01b4f280c69b31a786329af3165111eb7fbc4cbe6fc98d352341a87ad56c/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/51998d39-62ca-432f-bac3-3b8142d05e0e/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:212 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e882a774-f295-43dd-916c-e20c2df7f05a/volumes/kubernetes.io~projected/kube-api-access-4zzbc DeviceMajor:0 DeviceMinor:206 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/30ad9387-03e3-49d0-8ff8-b4959c6f343f/volumes/kubernetes.io~projected/kube-api-access-m8k7f DeviceMajor:0 DeviceMinor:61 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dc2a797ea8cd7857e843af4d0308e69f20e8de727faa0c34048bfab6331b9160/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/534103f4-2d9e-4bd7-ba78-d5e848d3924c/volumes/kubernetes.io~projected/kube-api-access-ptdph DeviceMajor:0 DeviceMinor:213 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/8369e5ee-521f-4922-9aff-7bcacc063cfa/volumes/kubernetes.io~projected/kube-api-access-jqmq6 DeviceMajor:0 DeviceMinor:222 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1ca946c4-34f5-4443-9ecf-e634055dacc0/volumes/kubernetes.io~projected/kube-api-access-mcrk5 DeviceMajor:0 DeviceMinor:64 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-220 DeviceMajor:0 DeviceMinor:220 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406011904 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/b94155ab-70f0-4efa-a01c-7303db0d8cb5/volumes/kubernetes.io~projected/kube-api-access-zcwq8 DeviceMajor:0 DeviceMinor:63 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67/volumes/kubernetes.io~projected/kube-api-access-px9wr DeviceMajor:0 DeviceMinor:65 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-171 DeviceMajor:0 DeviceMinor:171 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/56d9f680e2b84ec9503ce417aba7da15bd784cfabe763862db4173c37e8cebb4/userdata/shm DeviceMajor:0 DeviceMinor:223 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1ca946c4-34f5-4443-9ecf-e634055dacc0/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/1ca946c4-34f5-4443-9ecf-e634055dacc0/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0933120d5faf936a058dde92a7f114405b149969a72d2801c2b6e853f9d4260d/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9961b1778e0f5bc3f822cd88426734388acf849099e6fa143c63c8a29743d13e/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aad0ba21-8234-4c49-9c72-183b7b1dbcc5/volumes/kubernetes.io~projected/kube-api-access-gjg5x DeviceMajor:0 DeviceMinor:67 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/53e77f82-4b9d-4e97-9108-3f8f0740925f/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:227 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-87 DeviceMajor:0 DeviceMinor:87 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/87120866-99b9-46d6-9883-6244b38c38ae/volumes/kubernetes.io~projected/kube-api-access-kchqw DeviceMajor:0 DeviceMinor:59 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/f4fe5f4e-25f4-400c-9efb-f2f98757c74e/volumes/kubernetes.io~projected/kube-api-access-7xjbg DeviceMajor:0 DeviceMinor:62 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e14b06a2ee640758ad0d97a0eac3923c7a94e83289be79f1a5934374965e31ca/userdata/shm DeviceMajor:0 DeviceMinor:85 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/66634df91a7cd64cbc53723f4403b81d22922a9c62869f8cb686560f3a92f823/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/d91062aa-c9b2-48ac-865a-fc0acb25b4a6/volumes/kubernetes.io~projected/kube-api-access-vl6p9 DeviceMajor:0 DeviceMinor:197 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/efd51e149404d77dcc9115e9b25e48535536495dbfe3c0527247bd29ecc48f6a/userdata/shm DeviceMajor:0 DeviceMinor:214 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a0d45f8ac4f5cd045ae22cfd9aa7551c23f4597559d73253db976b70c3b3a030/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-231 DeviceMajor:0 DeviceMinor:231 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dfca312e-b151-4c3a-99bd-dd820925264b/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/bdf00c2a-249a-4bc8-af8d-89d2f080eb16/volumes/kubernetes.io~projected/kube-api-access-g8g55 DeviceMajor:0 DeviceMinor:60 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-141 DeviceMajor:0 DeviceMinor:141 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d91062aa-c9b2-48ac-865a-fc0acb25b4a6/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:56d9f680e2b84ec MacAddress:32:a8:24:1c:9d:e4 Speed:10000 Mtu:8901} {Name:a0d45f8ac4f5cd0 MacAddress:a2:7d:7d:3c:7a:6e Speed:10000 Mtu:8901} {Name:a6dc01b4f280c69 MacAddress:2a:80:7d:22:06:2a Speed:10000 Mtu:8901} {Name:a8c8dea2e42857e MacAddress:1e:e2:2f:8d:ee:f8 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:78:0a:f5:9f:0b Speed:0 Mtu:9001} {Name:br-int MacAddress:ba:6b:85:f9:c0:59 Speed:0 Mtu:8901} {Name:e3fcec2a2abff87 MacAddress:6e:38:35:ed:bc:67 Speed:10000 Mtu:8901} {Name:efd51e149404d77 MacAddress:ea:66:ab:e4:1f:b1 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:78:0a:f5:9f:0b Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:ea:34:aa:68:50:37 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:42:aa:02:66:5b:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812023808 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.251251    4512 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.251294    4512 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.251646    4512 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.251673    4512 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-191.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.251821    4512 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.251828    4512 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.251841    4512 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.251859    4512 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.251931    4512 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.252124    4512 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.252285    4512 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.252363    4512 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.252387    4512 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.252402    4512 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.252414    4512 kubelet.go:405] "Adding apiserver pod source"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.252424    4512 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.255210    4512 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.255244    4512 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:55:38.257150 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.256369    4512 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 01:55:38.259025 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259007    4512 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 01:55:38.259025 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259025    4512 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:55:38.259152 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259139    4512 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 01:55:38.259372 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259356    4512 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 01:55:38.259439 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259395    4512 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 01:55:38.259537 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259526    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 01:55:38.259599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259546    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 01:55:38.259599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259555    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 01:55:38.259599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259564    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 01:55:38.259599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259573    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 01:55:38.259599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259582    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 01:55:38.259599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259591    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 01:55:38.259599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259600    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 01:55:38.259897 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259611    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 01:55:38.259897 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259621    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 01:55:38.259897 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259634    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 01:55:38.259897 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259647    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 01:55:38.259897 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259682    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 01:55:38.259897 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.259692    4512 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 01:55:38.263100 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.263083    4512 server.go:1263] "Started kubelet"
Nov 04 01:55:38.263475 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.263192    4512 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 01:55:38.263475 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.263226    4512 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 01:55:38.263475 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.263295    4512 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 01:55:38.263665 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.263613    4512 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 01:55:38.263625 ip-10-0-142-191 systemd[1]: Started Kubernetes Kubelet.
Nov 04 01:55:38.264495 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.264477    4512 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 01:55:38.266301 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.266281    4512 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 01:55:38.266461 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.266428    4512 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 01:55:38.266574 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.266530    4512 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:49:59 +0000 UTC" deadline="2028-05-26 21:13:14.525867457 +0000 UTC"
Nov 04 01:55:38.266574 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.266560    4512 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13675h17m36.259310007s"
Nov 04 01:55:38.266769 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:38.266591    4512 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:55:38.266769 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.266627    4512 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 01:55:38.266769 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.266634    4512 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 01:55:38.266915 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.266871    4512 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 01:55:38.268714 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268679    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87120866-99b9-46d6-9883-6244b38c38ae" volumeName="kubernetes.io/projected/87120866-99b9-46d6-9883-6244b38c38ae-kube-api-access-kchqw" seLinuxMountContext=""
Nov 04 01:55:38.268791 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268719    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdf00c2a-249a-4bc8-af8d-89d2f080eb16" volumeName="kubernetes.io/configmap/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-iptables-alerter-script" seLinuxMountContext=""
Nov 04 01:55:38.268791 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268733    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dff859f4-bd6d-40b5-8a3d-fccf7e0baf67" volumeName="kubernetes.io/projected/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-kube-api-access-px9wr" seLinuxMountContext=""
Nov 04 01:55:38.268791 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268748    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7d3eb2e-9765-455d-b901-6596bdb56f7e" volumeName="kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-service-ca-bundle" seLinuxMountContext=""
Nov 04 01:55:38.268791 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268761    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7d3eb2e-9765-455d-b901-6596bdb56f7e" volumeName="kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 01:55:38.268791 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268775    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4fe5f4e-25f4-400c-9efb-f2f98757c74e" volumeName="kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cni-binary-copy" seLinuxMountContext=""
Nov 04 01:55:38.268791 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268787    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ca946c4-34f5-4443-9ecf-e634055dacc0" volumeName="kubernetes.io/projected/1ca946c4-34f5-4443-9ecf-e634055dacc0-kube-api-access-mcrk5" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268800    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c54f5f65-a78e-4dd3-908f-7242d386dd33" volumeName="kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-binary-copy" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268820    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30ad9387-03e3-49d0-8ff8-b4959c6f343f" volumeName="kubernetes.io/projected/30ad9387-03e3-49d0-8ff8-b4959c6f343f-kube-api-access-m8k7f" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268839    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b94155ab-70f0-4efa-a01c-7303db0d8cb5" volumeName="kubernetes.io/projected/b94155ab-70f0-4efa-a01c-7303db0d8cb5-kube-api-access-zcwq8" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268853    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdf00c2a-249a-4bc8-af8d-89d2f080eb16" volumeName="kubernetes.io/projected/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-kube-api-access-g8g55" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268866    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c54f5f65-a78e-4dd3-908f-7242d386dd33" volumeName="kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268878    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dfca312e-b151-4c3a-99bd-dd820925264b" volumeName="kubernetes.io/configmap/dfca312e-b151-4c3a-99bd-dd820925264b-konnectivity-ca" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268891    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4fe5f4e-25f4-400c-9efb-f2f98757c74e" volumeName="kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-daemon-config" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268906    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ca946c4-34f5-4443-9ecf-e634055dacc0" volumeName="kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-tuned" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268919    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="534103f4-2d9e-4bd7-ba78-d5e848d3924c" volumeName="kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268939    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53e77f82-4b9d-4e97-9108-3f8f0740925f" volumeName="kubernetes.io/configmap/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-cabundle" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268953    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53e77f82-4b9d-4e97-9108-3f8f0740925f" volumeName="kubernetes.io/projected/53e77f82-4b9d-4e97-9108-3f8f0740925f-kube-api-access-5cf6j" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268966    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87120866-99b9-46d6-9883-6244b38c38ae" volumeName="kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268978    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87120866-99b9-46d6-9883-6244b38c38ae" volumeName="kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-env-overrides" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.268990    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aad0ba21-8234-4c49-9c72-183b7b1dbcc5" volumeName="kubernetes.io/projected/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kube-api-access-gjg5x" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269004    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dfca312e-b151-4c3a-99bd-dd820925264b" volumeName="kubernetes.io/secret/dfca312e-b151-4c3a-99bd-dd820925264b-agent-certs" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269016    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dff859f4-bd6d-40b5-8a3d-fccf7e0baf67" volumeName="kubernetes.io/empty-dir/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-tmp-dir" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269028    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7d3eb2e-9765-455d-b901-6596bdb56f7e" volumeName="kubernetes.io/projected/e7d3eb2e-9765-455d-b901-6596bdb56f7e-kube-api-access-r9k72" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269041    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e882a774-f295-43dd-916c-e20c2df7f05a" volumeName="kubernetes.io/empty-dir/e882a774-f295-43dd-916c-e20c2df7f05a-data-volume" seLinuxMountContext=""
Nov 04 01:55:38.269057 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269054    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e882a774-f295-43dd-916c-e20c2df7f05a" volumeName="kubernetes.io/projected/e882a774-f295-43dd-916c-e20c2df7f05a-kube-api-access-4zzbc" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269066    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb7f58fe-db82-4c9e-bdfd-39725432e053" volumeName="kubernetes.io/empty-dir/eb7f58fe-db82-4c9e-bdfd-39725432e053-tmp-dir" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269080    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb7f58fe-db82-4c9e-bdfd-39725432e053" volumeName="kubernetes.io/projected/eb7f58fe-db82-4c9e-bdfd-39725432e053-kube-api-access-hjzd4" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269096    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4fe5f4e-25f4-400c-9efb-f2f98757c74e" volumeName="kubernetes.io/projected/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-kube-api-access-7xjbg" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269109    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51998d39-62ca-432f-bac3-3b8142d05e0e" volumeName="kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269121    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d91062aa-c9b2-48ac-865a-fc0acb25b4a6" volumeName="kubernetes.io/secret/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-cert" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269133    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb7f58fe-db82-4c9e-bdfd-39725432e053" volumeName="kubernetes.io/configmap/eb7f58fe-db82-4c9e-bdfd-39725432e053-config-volume" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269154    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ca946c4-34f5-4443-9ecf-e634055dacc0" volumeName="kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-tmp" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269167    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87120866-99b9-46d6-9883-6244b38c38ae" volumeName="kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-config" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269180    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d91062aa-c9b2-48ac-865a-fc0acb25b4a6" volumeName="kubernetes.io/projected/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-kube-api-access-vl6p9" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269193    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7d3eb2e-9765-455d-b901-6596bdb56f7e" volumeName="kubernetes.io/empty-dir/e7d3eb2e-9765-455d-b901-6596bdb56f7e-archives-path" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269204    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e882a774-f295-43dd-916c-e20c2df7f05a" volumeName="kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269218    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53e77f82-4b9d-4e97-9108-3f8f0740925f" volumeName="kubernetes.io/secret/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-key" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269230    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8369e5ee-521f-4922-9aff-7bcacc063cfa" volumeName="kubernetes.io/projected/8369e5ee-521f-4922-9aff-7bcacc063cfa-kube-api-access-jqmq6" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269242    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87120866-99b9-46d6-9883-6244b38c38ae" volumeName="kubernetes.io/secret/87120866-99b9-46d6-9883-6244b38c38ae-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269254    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b94155ab-70f0-4efa-a01c-7303db0d8cb5" volumeName="kubernetes.io/configmap/b94155ab-70f0-4efa-a01c-7303db0d8cb5-serviceca" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269269    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c54f5f65-a78e-4dd3-908f-7242d386dd33" volumeName="kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269280    4512 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c54f5f65-a78e-4dd3-908f-7242d386dd33" volumeName="kubernetes.io/projected/c54f5f65-a78e-4dd3-908f-7242d386dd33-kube-api-access-xdrl6" seLinuxMountContext=""
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269292    4512 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 01:55:38.269702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.269300    4512 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 01:55:38.270802 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.270741    4512 factory.go:55] Registering systemd factory
Nov 04 01:55:38.270802 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.270763    4512 factory.go:223] Registration of the systemd container factory successfully
Nov 04 01:55:38.275139 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.275122    4512 factory.go:153] Registering CRI-O factory
Nov 04 01:55:38.275139 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.275141    4512 factory.go:223] Registration of the crio container factory successfully
Nov 04 01:55:38.275254 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.275192    4512 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 01:55:38.275254 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.275209    4512 factory.go:103] Registering Raw factory
Nov 04 01:55:38.275254 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.275225    4512 manager.go:1199] Started watching for new ooms in manager
Nov 04 01:55:38.275620 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.275605    4512 manager.go:319] Starting recovery of all containers
Nov 04 01:55:38.302288 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.302242    4512 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 01:55:38.304156 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.304124    4512 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 01:55:38.304156 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.304151    4512 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 01:55:38.304314 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.304173    4512 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 01:55:38.304314 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:38.304217    4512 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 01:55:38.325120 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.325089    4512 generic.go:356] "Generic (PLEG): container finished" podID="23463ee86a57f13d80b429c8e67d6e17" containerID="746449b54e251e758b30d72993e52f67dc5f0bbeaa295b28b6d70fd85ffeee42" exitCode=0
Nov 04 01:55:38.329990 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.329957    4512 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="0ea2e4889a3b8c8f4551bd4a242cef3e55edceaa8226ec81aa5c32d882aa6870" exitCode=0
Nov 04 01:55:38.329990 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.329990    4512 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="d1433ef44defdfb7ae5aa38233ae7c474d6ef851a09581871a12a103c4e9f0a2" exitCode=0
Nov 04 01:55:38.330146 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.330001    4512 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="2ac8b1900815acec0d409e3960e8cde0f5cab35ec3f6bf010afff9802ebed08e" exitCode=0
Nov 04 01:55:38.330146 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.330013    4512 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="3d33afd7a73fc81601d301d8c409668cdf4c50e29636482ffed9e4840d34f4e0" exitCode=0
Nov 04 01:55:38.330146 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.330025    4512 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="67f262fbffd17fbf962e574290519da9573fed286dfabcd5dd62360e753fc3a3" exitCode=0
Nov 04 01:55:38.330146 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.330036    4512 generic.go:356] "Generic (PLEG): container finished" podID="c54f5f65-a78e-4dd3-908f-7242d386dd33" containerID="fb676d661078966974d3101e91ed770cf479e0cdd1ab52f5ba015fb5fe85e162" exitCode=0
Nov 04 01:55:38.347696 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.347667    4512 manager.go:324] Recovery completed
Nov 04 01:55:38.365093 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.365070    4512 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:38.367066 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:38.367041    4512 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:55:38.367424 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.367405    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:38.367498 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.367442    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:38.367498 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.367460    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:38.369008 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.368988    4512 cpu_manager.go:226] "Starting" policy="none"
Nov 04 01:55:38.369008 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369005    4512 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:38.369132 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369005    4512 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 01:55:38.369132 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369100    4512 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 01:55:38.369230 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369215    4512 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 01:55:38.369264 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369231    4512 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 01:55:38.369264 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369248    4512 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 01:55:38.369264 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369253    4512 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 01:55:38.369264 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369259    4512 policy_none.go:50] "Start"
Nov 04 01:55:38.369392 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369270    4512 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 01:55:38.369392 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369279    4512 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 01:55:38.369445 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369397    4512 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 01:55:38.369445 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369404    4512 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 01:55:38.369445 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369414    4512 policy_none.go:44] "Start"
Nov 04 01:55:38.369868 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369850    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:38.369940 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369880    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:38.369940 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.369891    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:38.373658 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.373618    4512 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 01:55:38.376549 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.376534    4512 manager.go:341] "Starting Device Plugin manager"
Nov 04 01:55:38.376613 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:38.376558    4512 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 01:55:38.376613 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.376566    4512 server.go:87] "Starting device plugin registration server"
Nov 04 01:55:38.376868 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.376857    4512 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 01:55:38.376899 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.376870    4512 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 01:55:38.376998 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.376977    4512 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 01:55:38.378173 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.377074    4512 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 01:55:38.378173 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.377115    4512 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 01:55:38.378173 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.377121    4512 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 01:55:38.378173 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.377166    4512 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:55:38.377102443Z","UUID":"b76576c2-f84c-4063-99f0-e9a245fa3564","Handler":null,"Name":"","Endpoint":""}
Nov 04 01:55:38.378379 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:38.378300    4512 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 01:55:38.378379 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:38.378347    4512 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-191.us-west-1.compute.internal\" not found"
Nov 04 01:55:38.378663 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.378634    4512 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 01:55:38.378663 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.378662    4512 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 01:55:38.405270 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.404494    4512 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"]
Nov 04 01:55:38.405408 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.405336    4512 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:38.406464 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.406442    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:38.406624 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.406475    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:38.406624 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.406491    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:38.406774 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.406761    4512 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:38.406892 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.406873    4512 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:38.407719 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.407673    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:38.407719 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.407706    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:38.407719 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.407717    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:38.407898 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.407804    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:38.407898 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.407831    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:38.407898 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.407844    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:38.408012 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.407977    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"9f88ddff8fbb738ea2fba74409a8de90","Type":"ContainerStarted","Data":"229db822cb3ffcd021eec334a6d9eab14f19664d43a0377399df88ef43f1f028"}
Nov 04 01:55:38.408048 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.408028    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"9f88ddff8fbb738ea2fba74409a8de90","Type":"ContainerStarted","Data":"a700b7f4b57af9ece70802725c34dec353f916ec71c513e5d3d68d54c4ed7f3e"}
Nov 04 01:55:38.408087 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.408067    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"23463ee86a57f13d80b429c8e67d6e17","Type":"ContainerStarted","Data":"7300cc1c00dac93c7538156cb14e9127caefba388ef0b1ff993aac7c78d542d8"}
Nov 04 01:55:38.408132 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.408088    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"23463ee86a57f13d80b429c8e67d6e17","Type":"ContainerDied","Data":"746449b54e251e758b30d72993e52f67dc5f0bbeaa295b28b6d70fd85ffeee42"}
Nov 04 01:55:38.408132 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.408108    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal" event={"ID":"23463ee86a57f13d80b429c8e67d6e17","Type":"ContainerStarted","Data":"b3eb0ec95449a0a40f3c1fb1812e3cec836189a48d2c75da55cf0a38c27799ee"}
Nov 04 01:55:38.408218 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.408132    4512 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:38.408218 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:38.407998    4512 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.409085 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.409067    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:38.409326 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.409307    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:38.409447 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.409331    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:38.409552 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:38.409533    4512 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-191.us-west-1.compute.internal\" not found" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.469015 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.468980    4512 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:55:38.473282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.470311    4512 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:55:38.473282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.470826    4512 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 01:55:38.477206 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.477144    4512 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:55:38.477311 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.477152    4512 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:55:38.478655 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.478637    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:55:38.478765 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.478667    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:55:38.478765 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.478681    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:55:38.478765 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.478722    4512 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.518964 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.518934    4512 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:55:38.569345 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.569314    4512 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.571462 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.571439    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.571591 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.571466    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.571591 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.571485    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f88ddff8fbb738ea2fba74409a8de90-config\") pod \"kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"9f88ddff8fbb738ea2fba74409a8de90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.611848 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.611818    4512 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.611955 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.611900    4512 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.612501 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.612475    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeNotReady"
Nov 04 01:55:38.612638 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.612524    4512 setters.go:546] "Node became not ready" node="ip-10-0-142-191.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-04T01:55:38Z","lastTransitionTime":"2026-11-04T01:55:38Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 04 01:55:38.629479 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.629461    4512 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:55:38.629587 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:38.629501    4512 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.629587 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.629526    4512 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.638653 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.638632    4512 nodeinfomanager.go:471] existing CSINode "ip-10-0-142-191.us-west-1.compute.internal" is owned by different node (oldNodeID="dc7b30d7-d966-41af-a4ae-b0a8d3db8810", newNodeID=""), cleaning up...
Nov 04 01:55:38.672467 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.672405    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f88ddff8fbb738ea2fba74409a8de90-config\") pod \"kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"9f88ddff8fbb738ea2fba74409a8de90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.672467 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.672433    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.672467 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.672452    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.672658 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.672501    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.672658 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.672532    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f88ddff8fbb738ea2fba74409a8de90-config\") pod \"kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"9f88ddff8fbb738ea2fba74409a8de90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.672658 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.672540    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23463ee86a57f13d80b429c8e67d6e17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" (UID: \"23463ee86a57f13d80b429c8e67d6e17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.693380 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.693356    4512 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:55:38.693485 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:38.693405    4512 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:38.881946 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.881912    4512 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-191.us-west-1.compute.internal" event="NodeReady"
Nov 04 01:55:38.882085 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:38.881989    4512 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 01:55:39.254063 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.254030    4512 apiserver.go:52] "Watching apiserver"
Nov 04 01:55:39.342165 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.342135    4512 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:39.342316 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.342289    4512 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:39.361775 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.361744    4512 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 01:55:39.362247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.362221    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4","kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal","openshift-multus/multus-mgw25","openshift-network-operator/iptables-alerter-rx5p4","openshift-service-ca/service-ca-5fffc97bdf-lql5t","openshift-multus/multus-additional-cni-plugins-c76wf","openshift-insights/periodic-gathering-lzc9g-h2w8g","openshift-network-diagnostics/network-check-target-jnspf","openshift-dns/node-resolver-tm85n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal","openshift-image-registry/node-ca-fgzvx","openshift-ovn-kubernetes/ovnkube-node-6gccn","openshift-cluster-node-tuning-operator/tuned-4k7bb","openshift-multus/network-metrics-daemon-6g7qh","kube-system/konnectivity-agent-5v5x7","openshift-insights/insights-runtime-extractor-s2nfk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm","openshift-dns/dns-default-l89qg","kube-system/global-pull-secret-syncer-dlxgx","openshift-ingress-canary/ingress-canary-qpbh2"]
Nov 04 01:55:39.362970 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.362950    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:39.369540 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.369520    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.370274 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.370256    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.376696 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.376674    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8g55\" (UniqueName: \"kubernetes.io/projected/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-kube-api-access-g8g55\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:55:39.376781 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.376705    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-kubelet-config\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:39.376781 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.376721    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-tmp\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.376781 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.376736    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.376902 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.376856    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-tmp\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.376993 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.376973    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-socket-dir-parent\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.377059 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377010    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-host-slash\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:55:39.377059 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377043    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcrk5\" (UniqueName: \"kubernetes.io/projected/1ca946c4-34f5-4443-9ecf-e634055dacc0-kube-api-access-mcrk5\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.377148 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377108    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-node-log\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.377218 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377197    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-env-overrides\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.377301 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377286    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-daemon-config\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.377352 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377314    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px9wr\" (UniqueName: \"kubernetes.io/projected/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-kube-api-access-px9wr\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:55:39.377352 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377339    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8k7f\" (UniqueName: \"kubernetes.io/projected/30ad9387-03e3-49d0-8ff8-b4959c6f343f-kube-api-access-m8k7f\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:39.377446 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377365    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-sys-fs\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.377446 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377392    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-modprobe-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.377446 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377419    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87120866-99b9-46d6-9883-6244b38c38ae-ovn-node-metrics-cert\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.377599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377445    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-netns\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.377599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377530    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-kubelet\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.377599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377557    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:39.377729 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377614    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-sys\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.377729 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377640    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-var-lib-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.377729 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377666    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-config\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.377855 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377780    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kchqw\" (UniqueName: \"kubernetes.io/projected/87120866-99b9-46d6-9883-6244b38c38ae-kube-api-access-kchqw\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.377855 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377814    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-conf-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.377855 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377836    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-multus-certs\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.377990 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377853    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-iptables-alerter-script\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:55:39.377990 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377885    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-bin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.377990 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377928    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.377990 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.377967    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xjbg\" (UniqueName: \"kubernetes.io/projected/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-kube-api-access-7xjbg\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.378148 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378054    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdrl6\" (UniqueName: \"kubernetes.io/projected/c54f5f65-a78e-4dd3-908f-7242d386dd33-kube-api-access-xdrl6\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.378204 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378193    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcwq8\" (UniqueName: \"kubernetes.io/projected/b94155ab-70f0-4efa-a01c-7303db0d8cb5-kube-api-access-zcwq8\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:55:39.378255 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378220    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjg5x\" (UniqueName: \"kubernetes.io/projected/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kube-api-access-gjg5x\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.378255 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378245    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.378379 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378360    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-multus\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.378471 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378439    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-var-lib-kubelet\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.378539 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378494    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cnibin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.378595 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378535    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b94155ab-70f0-4efa-a01c-7303db0d8cb5-host\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:55:39.378595 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378559    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-socket-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.378712 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378595    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-systemd\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.378712 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378637    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-hostroot\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.378712 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378682    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b94155ab-70f0-4efa-a01c-7303db0d8cb5-serviceca\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:55:39.378806 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378746    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-lib-modules\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.378856 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378809    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-host\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.378856 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378830    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-systemd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.378952 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378871    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-log-socket\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.378952 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378902    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-netd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.378952 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378937    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.379098 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378962    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-etc-kubernetes\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.379098 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.378991    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.379098 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379041    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-conf\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.379098 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379085    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-system-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.379275 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379109    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.379275 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379138    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.379275 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379171    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.379275 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379201    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfca312e-b151-4c3a-99bd-dd820925264b-konnectivity-ca\") pod \"konnectivity-agent-5v5x7\" (UID: \"dfca312e-b151-4c3a-99bd-dd820925264b\") " pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:55:39.379275 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379230    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:39.379275 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379258    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-slash\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.379581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379311    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-script-lib\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.379581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379362    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-cnibin\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.379581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379392    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-registration-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.379581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379416    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysconfig\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.379581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379447    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-kubernetes\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.379581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379469    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-bin\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.379581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379495    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-dbus\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:39.379581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379543    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.379581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379579    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-run\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379627    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-kubelet\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379650    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-systemd-units\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379683    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-netns\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379706    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-etc-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379733    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-ovn\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379802    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-device-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379837    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-tuned\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379866    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cni-binary-copy\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379899    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-k8s-cni-cncf-io\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379927    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379951    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfca312e-b151-4c3a-99bd-dd820925264b-agent-certs\") pod \"konnectivity-agent-5v5x7\" (UID: \"dfca312e-b151-4c3a-99bd-dd820925264b\") " pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:55:39.379986 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.379983    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-tmp-dir\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:55:39.380532 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.380012    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:39.380532 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.380038    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-system-cni-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.380532 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.380064    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-binary-copy\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.380532 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.380092    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-hosts-file\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:55:39.380532 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.380347    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-tuned\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.380532 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.380473    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-tmp-dir\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:55:39.423698 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.423671    4512 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:55:39.423824 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:39.423721    4512 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:39.425089 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.425070    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 01:55:39.425329 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.425307    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:39.428633 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.428612    4512 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:55:39.428695 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:39.428660    4512 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-191.us-west-1.compute.internal"
Nov 04 01:55:39.429081 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.429067    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b94155ab-70f0-4efa-a01c-7303db0d8cb5-serviceca\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:55:39.436135 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.436111    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zhvgj\""
Nov 04 01:55:39.436254 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.436211    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bxxxw\""
Nov 04 01:55:39.436254 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.436252    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 01:55:39.436330 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.436289    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tlvkz\""
Nov 04 01:55:39.436404 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.436392    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s4jjj\""
Nov 04 01:55:39.436476 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.436467    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r7nnn\""
Nov 04 01:55:39.436552 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.436542    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bzrbt\""
Nov 04 01:55:39.436714 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.436682    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:55:39.436858 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.436833    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nrwjt\""
Nov 04 01:55:39.436980 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.436962    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zzzll\""
Nov 04 01:55:39.437073 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.437051    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:55:39.437170 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.437072    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wpjt7\""
Nov 04 01:55:39.437170 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.437091    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zk98g\""
Nov 04 01:55:39.437170 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.437134    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tvbnh\""
Nov 04 01:55:39.437289 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.437183    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 01:55:39.437289 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.437271    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7qkcb\""
Nov 04 01:55:39.437761 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.437747    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qpbh2_d91062aa-c9b2-48ac-865a-fc0acb25b4a6/serve-healthcheck-canary/0.log"
Nov 04 01:55:39.440997 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.440982    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51998d39-62ca-432f-bac3-3b8142d05e0e-original-pull-secret\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:39.441039 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.441018    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfca312e-b151-4c3a-99bd-dd820925264b-agent-certs\") pod \"konnectivity-agent-5v5x7\" (UID: \"dfca312e-b151-4c3a-99bd-dd820925264b\") " pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:55:39.453655 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.453637    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 01:55:39.453716 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.453705    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 01:55:39.454432 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.454406    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 01:55:39.454534 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.454522    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-9ldb4\""
Nov 04 01:55:39.457696 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.457680    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87120866-99b9-46d6-9883-6244b38c38ae-ovn-node-metrics-cert\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.468396 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.468378    4512 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 01:55:39.480083 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480063    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.480186 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480129    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.480262 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480228    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.480262 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480233    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.480332 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480288    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.480369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480344    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-cert\") pod \"ingress-canary-qpbh2\" (UID: \"d91062aa-c9b2-48ac-865a-fc0acb25b4a6\") " pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:39.480369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480362    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-sys-fs\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.480465 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480378    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-modprobe-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.480465 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480390    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.480465 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480403    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-netns\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.480465 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480437    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 01:55:39.480465 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480438    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-kubelet\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.480697 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480472    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.480697 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480518    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.480697 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480500    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-netns\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.480697 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480490    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-kubelet\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.480697 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480550    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-modprobe-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.480697 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480553    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 01:55:39.480697 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480603    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-service-ca-bundle\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:39.480697 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480603    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-sys-fs\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.481002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480725    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-sys\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.481002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480756    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-sys\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.481002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480751    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-var-lib-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.481002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480789    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-conf-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480806    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-var-lib-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.481002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480859    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-multus-certs\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480809    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-multus-certs\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480907    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.481002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480908    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-cert\") pod \"ingress-canary-qpbh2\" (UID: \"d91062aa-c9b2-48ac-865a-fc0acb25b4a6\") " pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:39.481002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.480968    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-conf-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481009    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-bin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481060    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-bin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481098    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481150    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqmq6\" (UniqueName: \"kubernetes.io/projected/8369e5ee-521f-4922-9aff-7bcacc063cfa-kube-api-access-jqmq6\") pod \"migrator-66df5d496b-6xdt4\" (UID: \"8369e5ee-521f-4922-9aff-7bcacc063cfa\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4"
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481170    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481181    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e7d3eb2e-9765-455d-b901-6596bdb56f7e-archives-path\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481211    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481234    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e7d3eb2e-9765-455d-b901-6596bdb56f7e-archives-path\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481239    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481236    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-multus\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481272    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-var-lib-kubelet\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481291    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-var-lib-cni-multus\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481294    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cnibin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481274    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481327    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b94155ab-70f0-4efa-a01c-7303db0d8cb5-host\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481337    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-var-lib-kubelet\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481356    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9k72\" (UniqueName: \"kubernetes.io/projected/e7d3eb2e-9765-455d-b901-6596bdb56f7e-kube-api-access-r9k72\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481365    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cnibin\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481376    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b94155ab-70f0-4efa-a01c-7303db0d8cb5-host\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481386    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-cabundle\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481404    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-socket-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481424    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-systemd\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481439    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-hostroot\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481462    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-trusted-ca-bundle\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481485    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-lib-modules\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481520    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-host\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481498    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-systemd\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481547    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-hostroot\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481551    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-systemd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481580    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-systemd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481585    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-cabundle\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481590    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-socket-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481646    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-host\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481693    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-log-socket\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481709    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-lib-modules\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481738    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-log-socket\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.481759 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481756    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-netd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481789    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481812    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-etc-kubernetes\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481835    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481840    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-netd\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481853    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481884    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-etc-kubernetes\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481877    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-conf\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481912    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-system-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481925    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481953    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481967    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-slash\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481984    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-d\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481992    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysctl-conf\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.481996    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-slash\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482001    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-system-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482022    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-cni-dir\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482064    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-cnibin\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482090    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482134    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zzbc\" (UniqueName: \"kubernetes.io/projected/e882a774-f295-43dd-916c-e20c2df7f05a-kube-api-access-4zzbc\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482167    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-cnibin\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482167    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb7f58fe-db82-4c9e-bdfd-39725432e053-tmp-dir\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482225    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjzd4\" (UniqueName: \"kubernetes.io/projected/eb7f58fe-db82-4c9e-bdfd-39725432e053-kube-api-access-hjzd4\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482238    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb7f58fe-db82-4c9e-bdfd-39725432e053-tmp-dir\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482250    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-registration-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482272    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysconfig\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482294    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-kubernetes\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482299    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-registration-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482288    4512 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482344    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-sysconfig\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482352    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-etc-kubernetes\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482355    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-bin\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482383    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-cni-bin\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482385    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-dbus\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482407    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482414    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-dbus\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482428    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-run\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482464    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482453    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-kubelet\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482499    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-systemd-units\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482500    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-kubelet\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482501    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1ca946c4-34f5-4443-9ecf-e634055dacc0-run\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482540    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-netns\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482573    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-etc-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482576    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-netns\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482598    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-ovn\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482609    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-etc-openvswitch\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482670    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-systemd-units\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482669    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-run-ovn\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482694    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-device-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482726    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-device-dir\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.482703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482727    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-k8s-cni-cncf-io\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482760    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482777    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-host-run-k8s-cni-cncf-io\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482814    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482845    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e882a774-f295-43dd-916c-e20c2df7f05a-crio-socket\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.482978    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-system-cni-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483003    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-hosts-file\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483037    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb7f58fe-db82-4c9e-bdfd-39725432e053-config-volume\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483061    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c54f5f65-a78e-4dd3-908f-7242d386dd33-system-cni-dir\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483060    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483098    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-hosts-file\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483104    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl6p9\" (UniqueName: \"kubernetes.io/projected/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-kube-api-access-vl6p9\") pod \"ingress-canary-qpbh2\" (UID: \"d91062aa-c9b2-48ac-865a-fc0acb25b4a6\") " pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483134    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-kubelet-config\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483163    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483192    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483194    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51998d39-62ca-432f-bac3-3b8142d05e0e-kubelet-config\") pod \"global-pull-secret-syncer-dlxgx\" (UID: \"51998d39-62ca-432f-bac3-3b8142d05e0e\") " pod="kube-system/global-pull-secret-syncer-dlxgx"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483293    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-socket-dir-parent\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483315    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-host-slash\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483330    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e882a774-f295-43dd-916c-e20c2df7f05a-data-volume\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483344    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-key\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483374    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cf6j\" (UniqueName: \"kubernetes.io/projected/53e77f82-4b9d-4e97-9108-3f8f0740925f-kube-api-access-5cf6j\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483386    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-host-slash\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483401    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-node-log\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483421    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e882a774-f295-43dd-916c-e20c2df7f05a-data-volume\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483428    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-socket-dir-parent\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.484168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.483448    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87120866-99b9-46d6-9883-6244b38c38ae-node-log\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.484818 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.484606    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ad9387-03e3-49d0-8ff8-b4959c6f343f-metrics-certs\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:39.484982 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.484898    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 01:55:39.484982 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.484928    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.484982 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.484927    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.484982 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.484941    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 01:55:39.484982 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.484944    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.485198 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485073    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.485198 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485083    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.485198 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485078    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 01:55:39.485198 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485088    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.485345 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485244    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.485345 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485252    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 01:55:39.485345 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485260    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 01:55:39.485345 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485263    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.485345 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485297    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.485345 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485307    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 01:55:39.485345 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485325    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 01:55:39.485345 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485329    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.485638 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485615    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 01:55:39.485678 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485636    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.485784 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485765    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 01:55:39.485913 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485896    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.485952 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485901    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.485991 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485954    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.485991 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.485970    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 01:55:39.488352 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.488336    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-config\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.488421 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.488373    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-env-overrides\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.488421 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.488379    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-multus-daemon-config\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.489790 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.489772    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdrl6\" (UniqueName: \"kubernetes.io/projected/c54f5f65-a78e-4dd3-908f-7242d386dd33-kube-api-access-xdrl6\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.490248 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.490220    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcwq8\" (UniqueName: \"kubernetes.io/projected/b94155ab-70f0-4efa-a01c-7303db0d8cb5-kube-api-access-zcwq8\") pod \"node-ca-fgzvx\" (UID: \"b94155ab-70f0-4efa-a01c-7303db0d8cb5\") " pod="openshift-image-registry/node-ca-fgzvx"
Nov 04 01:55:39.490304 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.490287    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptdph\" (UniqueName: \"kubernetes.io/projected/534103f4-2d9e-4bd7-ba78-d5e848d3924c-kube-api-access-ptdph\") pod \"network-check-target-jnspf\" (UID: \"534103f4-2d9e-4bd7-ba78-d5e848d3924c\") " pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:39.490349 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.490288    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kchqw\" (UniqueName: \"kubernetes.io/projected/87120866-99b9-46d6-9883-6244b38c38ae-kube-api-access-kchqw\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.491286 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.490726    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.491286 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.490818    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfca312e-b151-4c3a-99bd-dd820925264b-konnectivity-ca\") pod \"konnectivity-agent-5v5x7\" (UID: \"dfca312e-b151-4c3a-99bd-dd820925264b\") " pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:55:39.491286 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.490821    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-cni-binary-copy\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.491286 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.490864    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.491286 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.490910    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-service-ca-bundle\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:39.491286 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.490991    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c54f5f65-a78e-4dd3-908f-7242d386dd33-cni-binary-copy\") pod \"multus-additional-cni-plugins-c76wf\" (UID: \"c54f5f65-a78e-4dd3-908f-7242d386dd33\") " pod="openshift-multus/multus-additional-cni-plugins-c76wf"
Nov 04 01:55:39.491623 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.491368    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcrk5\" (UniqueName: \"kubernetes.io/projected/1ca946c4-34f5-4443-9ecf-e634055dacc0-kube-api-access-mcrk5\") pod \"tuned-4k7bb\" (UID: \"1ca946c4-34f5-4443-9ecf-e634055dacc0\") " pod="openshift-cluster-node-tuning-operator/tuned-4k7bb"
Nov 04 01:55:39.494148 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.493881    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb7f58fe-db82-4c9e-bdfd-39725432e053-config-volume\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.494235 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.494222    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e882a774-f295-43dd-916c-e20c2df7f05a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.494479 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.494454    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xjbg\" (UniqueName: \"kubernetes.io/projected/f4fe5f4e-25f4-400c-9efb-f2f98757c74e-kube-api-access-7xjbg\") pod \"multus-mgw25\" (UID: \"f4fe5f4e-25f4-400c-9efb-f2f98757c74e\") " pod="openshift-multus/multus-mgw25"
Nov 04 01:55:39.494594 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.494572    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8g55\" (UniqueName: \"kubernetes.io/projected/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-kube-api-access-g8g55\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:55:39.494642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.494588    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-px9wr\" (UniqueName: \"kubernetes.io/projected/dff859f4-bd6d-40b5-8a3d-fccf7e0baf67-kube-api-access-px9wr\") pod \"node-resolver-tm85n\" (UID: \"dff859f4-bd6d-40b5-8a3d-fccf7e0baf67\") " pod="openshift-dns/node-resolver-tm85n"
Nov 04 01:55:39.494642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.494572    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjg5x\" (UniqueName: \"kubernetes.io/projected/aad0ba21-8234-4c49-9c72-183b7b1dbcc5-kube-api-access-gjg5x\") pod \"aws-ebs-csi-driver-node-c4mzm\" (UID: \"aad0ba21-8234-4c49-9c72-183b7b1dbcc5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4mzm"
Nov 04 01:55:39.495585 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.495565    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8k7f\" (UniqueName: \"kubernetes.io/projected/30ad9387-03e3-49d0-8ff8-b4959c6f343f-kube-api-access-m8k7f\") pod \"network-metrics-daemon-6g7qh\" (UID: \"30ad9387-03e3-49d0-8ff8-b4959c6f343f\") " pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:39.503251 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.503237    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 01:55:39.510353 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.510317    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 01:55:39.510446 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.510431    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87120866-99b9-46d6-9883-6244b38c38ae-ovnkube-script-lib\") pod \"ovnkube-node-6gccn\" (UID: \"87120866-99b9-46d6-9883-6244b38c38ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:39.512700 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.512686    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 01:55:39.518891 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.518873    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bdf00c2a-249a-4bc8-af8d-89d2f080eb16-iptables-alerter-script\") pod \"iptables-alerter-rx5p4\" (UID: \"bdf00c2a-249a-4bc8-af8d-89d2f080eb16\") " pod="openshift-network-operator/iptables-alerter-rx5p4"
Nov 04 01:55:39.522265 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.522248    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-trusted-ca-bundle\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:39.528205 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.528190    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 04 01:55:39.528280 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.528256    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-c7hth\""
Nov 04 01:55:39.528347 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.528328    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rrqh4\""
Nov 04 01:55:39.528376 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.528354    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k89rs\""
Nov 04 01:55:39.528402 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.528393    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 01:55:39.528438 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.528399    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mj9m5\""
Nov 04 01:55:39.528438 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.528415    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6wh47\""
Nov 04 01:55:39.534042 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.534025    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/53e77f82-4b9d-4e97-9108-3f8f0740925f-signing-key\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:39.539994 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.539975    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqmq6\" (UniqueName: \"kubernetes.io/projected/8369e5ee-521f-4922-9aff-7bcacc063cfa-kube-api-access-jqmq6\") pod \"migrator-66df5d496b-6xdt4\" (UID: \"8369e5ee-521f-4922-9aff-7bcacc063cfa\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-6xdt4"
Nov 04 01:55:39.541149 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.541133    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjzd4\" (UniqueName: \"kubernetes.io/projected/eb7f58fe-db82-4c9e-bdfd-39725432e053-kube-api-access-hjzd4\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.541371 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.541355    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9k72\" (UniqueName: \"kubernetes.io/projected/e7d3eb2e-9765-455d-b901-6596bdb56f7e-kube-api-access-r9k72\") pod \"periodic-gathering-lzc9g-h2w8g\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") " pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:55:39.542676 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.542661    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl6p9\" (UniqueName: \"kubernetes.io/projected/d91062aa-c9b2-48ac-865a-fc0acb25b4a6-kube-api-access-vl6p9\") pod \"ingress-canary-qpbh2\" (UID: \"d91062aa-c9b2-48ac-865a-fc0acb25b4a6\") " pod="openshift-ingress-canary/ingress-canary-qpbh2"
Nov 04 01:55:39.546108 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.546091    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cf6j\" (UniqueName: \"kubernetes.io/projected/53e77f82-4b9d-4e97-9108-3f8f0740925f-kube-api-access-5cf6j\") pod \"service-ca-5fffc97bdf-lql5t\" (UID: \"53e77f82-4b9d-4e97-9108-3f8f0740925f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-lql5t"
Nov 04 01:55:39.546244 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.546226    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zzbc\" (UniqueName: \"kubernetes.io/projected/e882a774-f295-43dd-916c-e20c2df7f05a-kube-api-access-4zzbc\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.583968 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.583935    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e882a774-f295-43dd-916c-e20c2df7f05a-crio-socket\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.583968 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.583966    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.584166 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.583991    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.584166 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.584054    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e882a774-f295-43dd-916c-e20c2df7f05a-crio-socket\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:39.584166 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:39.584108    4512 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:39.584306 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:39.584183    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:40.084169401 +0000 UTC m=+1.889626030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:39.586473 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.586446    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb7f58fe-db82-4c9e-bdfd-39725432e053-metrics-tls\") pod \"dns-default-l89qg\" (UID: \"eb7f58fe-db82-4c9e-bdfd-39725432e053\") " pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.664052 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.664023    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6g7qh"
Nov 04 01:55:39.669827 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.669803    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:39.831977 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.831952    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-6xdt4_8369e5ee-521f-4922-9aff-7bcacc063cfa/migrator/0.log"
Nov 04 01:55:39.938142 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:39.938112    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:40.052498 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:40.052416    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6g7qh"]
Nov 04 01:55:40.087481 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:40.087445    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:40.087677 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:40.087592    4512 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:40.087677 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:40.087672    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:41.087654379 +0000 UTC m=+2.893111001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:40.093766 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:40.093741    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ad9387_03e3_49d0_8ff8_b4959c6f343f.slice/crio-4df634ede36ad5900be19e87a537f17ff18ce20b2d9a9390034df770d4511c6f WatchSource:0}: Error finding container 4df634ede36ad5900be19e87a537f17ff18ce20b2d9a9390034df770d4511c6f: Status 404 returned error can't find the container with id 4df634ede36ad5900be19e87a537f17ff18ce20b2d9a9390034df770d4511c6f
Nov 04 01:55:40.095940 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:40.095923    4512 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:55:40.103994 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:40.103923    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l89qg"]
Nov 04 01:55:40.107351 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:40.107330    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7f58fe_db82_4c9e_bdfd_39725432e053.slice/crio-2556825c72565f330077083814438afb9e2ed3fb1e337a2bc9d8396d0f3f825d WatchSource:0}: Error finding container 2556825c72565f330077083814438afb9e2ed3fb1e337a2bc9d8396d0f3f825d: Status 404 returned error can't find the container with id 2556825c72565f330077083814438afb9e2ed3fb1e337a2bc9d8396d0f3f825d
Nov 04 01:55:40.141939 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:40.141914    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:40.347220 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:40.347142    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l89qg" event={"ID":"eb7f58fe-db82-4c9e-bdfd-39725432e053","Type":"ContainerStarted","Data":"2556825c72565f330077083814438afb9e2ed3fb1e337a2bc9d8396d0f3f825d"}
Nov 04 01:55:40.348321 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:40.348296    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6g7qh" event={"ID":"30ad9387-03e3-49d0-8ff8-b4959c6f343f","Type":"ContainerStarted","Data":"4df634ede36ad5900be19e87a537f17ff18ce20b2d9a9390034df770d4511c6f"}
Nov 04 01:55:40.462661 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:40.462623    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:40.670618 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:40.670531    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:40.870781 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:40.870742    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:41.094463 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:41.094427    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:41.094637 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:41.094599    4512 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:41.094680 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:41.094670    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:43.094655406 +0000 UTC m=+4.900112031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:41.262625 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:41.262581    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:42.017985 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:42.017961    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5v5x7"
Nov 04 01:55:42.063204 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:42.063171    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:42.355603 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:42.355563    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6g7qh" event={"ID":"30ad9387-03e3-49d0-8ff8-b4959c6f343f","Type":"ContainerStarted","Data":"d58c536d01fab3caf18175402e46de34e5df1d770e420d5e981512a6898455b1"}
Nov 04 01:55:42.355790 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:42.355622    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6g7qh" event={"ID":"30ad9387-03e3-49d0-8ff8-b4959c6f343f","Type":"ContainerStarted","Data":"115aea3bbaa2127e1de23b1973a47c77877368bb81006cc66504d9c13fd3eb17"}
Nov 04 01:55:42.460991 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:42.460913    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:42.707624 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:42.707583    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:42.912341 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:42.912308    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:43.105432 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:43.105407    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:43.105761 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:43.105541    4512 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:43.105761 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:43.105598    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls podName:e882a774-f295-43dd-916c-e20c2df7f05a nodeName:}" failed. No retries permitted until 2026-11-04 01:55:47.105581163 +0000 UTC m=+8.911037773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2nfk" (UID: "e882a774-f295-43dd-916c-e20c2df7f05a") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:55:43.262798 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:43.262767    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:43.360731 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:43.360688    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l89qg" event={"ID":"eb7f58fe-db82-4c9e-bdfd-39725432e053","Type":"ContainerStarted","Data":"ec51fa04159b218074abc50965ed9797ab497555af5529e05d39aaee96924460"}
Nov 04 01:55:43.360898 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:43.360774    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l89qg" event={"ID":"eb7f58fe-db82-4c9e-bdfd-39725432e053","Type":"ContainerStarted","Data":"576eb1e663c14ddccce288e9a864da303ac2cb2a0ecf6aadb79dc077c479913f"}
Nov 04 01:55:43.360967 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:43.360937    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:43.469328 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:43.469245    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:43.579385 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:43.579357    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:44.066090 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:44.066038    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:44.466769 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:44.466696    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:44.720862 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:44.720790    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:44.928957 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:44.928922    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:45.259436 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:45.259400    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:46.068757 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:46.068725    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:46.457574 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:46.457479    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:46.658936 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:46.658901    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:46.894470 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:46.894428    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:47.096182 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:47.096146    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:47.130335 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:47.130302    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:47.132727 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:47.132701    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e882a774-f295-43dd-916c-e20c2df7f05a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2nfk\" (UID: \"e882a774-f295-43dd-916c-e20c2df7f05a\") " pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:47.170979 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:47.170900    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s2nfk"
Nov 04 01:55:47.297524 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:47.297478    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:47.487144 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:47.487116    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:47.487315 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:47.487304    4512 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:55:47.544728 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:47.544704    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode882a774_f295_43dd_916c_e20c2df7f05a.slice/crio-54eed488780d3c671961dac8ee8a8acb52ee094472e1ea02cc32ceaf08f26eba WatchSource:0}: Error finding container 54eed488780d3c671961dac8ee8a8acb52ee094472e1ea02cc32ceaf08f26eba: Status 404 returned error can't find the container with id 54eed488780d3c671961dac8ee8a8acb52ee094472e1ea02cc32ceaf08f26eba
Nov 04 01:55:47.591039 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:47.591011    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s2nfk"]
Nov 04 01:55:47.664040 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:47.664006    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:55:48.062706 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:48.062670    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:48.378728 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:48.378629    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2nfk" event={"ID":"e882a774-f295-43dd-916c-e20c2df7f05a","Type":"ContainerStarted","Data":"c4d83f42be049fdc6a1cee5c6b7dc7ac5394558fbba8c161db657f1d65af7771"}
Nov 04 01:55:48.378728 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:48.378690    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2nfk" event={"ID":"e882a774-f295-43dd-916c-e20c2df7f05a","Type":"ContainerStarted","Data":"54eed488780d3c671961dac8ee8a8acb52ee094472e1ea02cc32ceaf08f26eba"}
Nov 04 01:55:48.471715 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:48.471672    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:48.718333 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:48.718307    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:48.921614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:48.921582    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:49.123493 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:49.123454    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:49.325169 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:49.325133    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:49.383363 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:49.383286    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2nfk" event={"ID":"e882a774-f295-43dd-916c-e20c2df7f05a","Type":"ContainerStarted","Data":"cb275a0d2a8fc5d494ea7837ca6a1c2e94dbb850300b3dcec153f14059813047"}
Nov 04 01:55:50.391998 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:50.391962    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:50.643879 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:50.643773    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:50.837040 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:50.837001    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:51.155916 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:51.155873    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:51.390587 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:51.390550    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2nfk" event={"ID":"e882a774-f295-43dd-916c-e20c2df7f05a","Type":"ContainerStarted","Data":"ca67621d69688c6a7f3f31a6f62a58209385407a817f7dc72f96e47edf2fe12d"}
Nov 04 01:55:51.405117 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:51.405084    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:51.604518 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:51.604484    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:52.346628 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.346599    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-dblsg"]
Nov 04 01:55:52.348341 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.348325    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-dblsg"
Nov 04 01:55:52.403333 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.403304    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 01:55:52.427598 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.427572    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 01:55:52.427936 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.427664    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-89hgq\""
Nov 04 01:55:52.453031 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.453003    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-dblsg"]
Nov 04 01:55:52.468229 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.468196    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwqcd\" (UniqueName: \"kubernetes.io/projected/49445752-7958-49b2-afb3-2c04bc41ec3a-kube-api-access-pwqcd\") pod \"downloads-6dd789c8b7-dblsg\" (UID: \"49445752-7958-49b2-afb3-2c04bc41ec3a\") " pod="openshift-console/downloads-6dd789c8b7-dblsg"
Nov 04 01:55:52.468229 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.468232    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49445752-7958-49b2-afb3-2c04bc41ec3a-tmp\") pod \"downloads-6dd789c8b7-dblsg\" (UID: \"49445752-7958-49b2-afb3-2c04bc41ec3a\") " pod="openshift-console/downloads-6dd789c8b7-dblsg"
Nov 04 01:55:52.568902 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.568873    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwqcd\" (UniqueName: \"kubernetes.io/projected/49445752-7958-49b2-afb3-2c04bc41ec3a-kube-api-access-pwqcd\") pod \"downloads-6dd789c8b7-dblsg\" (UID: \"49445752-7958-49b2-afb3-2c04bc41ec3a\") " pod="openshift-console/downloads-6dd789c8b7-dblsg"
Nov 04 01:55:52.568902 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.568904    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49445752-7958-49b2-afb3-2c04bc41ec3a-tmp\") pod \"downloads-6dd789c8b7-dblsg\" (UID: \"49445752-7958-49b2-afb3-2c04bc41ec3a\") " pod="openshift-console/downloads-6dd789c8b7-dblsg"
Nov 04 01:55:52.569267 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.569222    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49445752-7958-49b2-afb3-2c04bc41ec3a-tmp\") pod \"downloads-6dd789c8b7-dblsg\" (UID: \"49445752-7958-49b2-afb3-2c04bc41ec3a\") " pod="openshift-console/downloads-6dd789c8b7-dblsg"
Nov 04 01:55:52.591213 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.591179    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:52.630230 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.630168    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwqcd\" (UniqueName: \"kubernetes.io/projected/49445752-7958-49b2-afb3-2c04bc41ec3a-kube-api-access-pwqcd\") pod \"downloads-6dd789c8b7-dblsg\" (UID: \"49445752-7958-49b2-afb3-2c04bc41ec3a\") " pod="openshift-console/downloads-6dd789c8b7-dblsg"
Nov 04 01:55:52.658501 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.658475    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-dblsg"
Nov 04 01:55:52.788112 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:52.788080    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:53.039675 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:53.039637    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:53.133199 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:53.133170    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-dblsg"]
Nov 04 01:55:53.136436 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:53.136407    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49445752_7958_49b2_afb3_2c04bc41ec3a.slice/crio-81c7ac9c6491d39047558a868617a525bf2ed540e3edf74864fde6a144008d2b WatchSource:0}: Error finding container 81c7ac9c6491d39047558a868617a525bf2ed540e3edf74864fde6a144008d2b: Status 404 returned error can't find the container with id 81c7ac9c6491d39047558a868617a525bf2ed540e3edf74864fde6a144008d2b
Nov 04 01:55:53.241599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:53.241564    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:53.365810 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:53.365739    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l89qg"
Nov 04 01:55:53.397285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:53.397245    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-dblsg" event={"ID":"49445752-7958-49b2-afb3-2c04bc41ec3a","Type":"ContainerStarted","Data":"81c7ac9c6491d39047558a868617a525bf2ed540e3edf74864fde6a144008d2b"}
Nov 04 01:55:53.787492 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:53.787457    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:53.988889 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:53.988848    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:54.359573 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:54.359523    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:54.568029 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:54.567984    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:56.776470 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:55:56.776438    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 01:55:57.907243 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:57.907209    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:58.164374 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:58.164300    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:58.957565 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:58.957526    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:58.988595 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:58.988561    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s"]
Nov 04 01:55:58.990844 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:58.990819    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s"
Nov 04 01:55:59.043176 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:59.043139    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 04 01:55:59.092402 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:59.092365    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-5zrx6\""
Nov 04 01:55:59.093216 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:59.093186    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s"]
Nov 04 01:55:59.118629 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:59.118602    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0f437497-7f45-4df4-a309-7bdd9bb697c4-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-2526s\" (UID: \"0f437497-7f45-4df4-a309-7bdd9bb697c4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s"
Nov 04 01:55:59.219265 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:59.219170    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0f437497-7f45-4df4-a309-7bdd9bb697c4-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-2526s\" (UID: \"0f437497-7f45-4df4-a309-7bdd9bb697c4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s"
Nov 04 01:55:59.221945 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:59.221919    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0f437497-7f45-4df4-a309-7bdd9bb697c4-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-2526s\" (UID: \"0f437497-7f45-4df4-a309-7bdd9bb697c4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s"
Nov 04 01:55:59.248356 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:59.248328    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gccn"
Nov 04 01:55:59.305068 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:59.305036    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s"
Nov 04 01:55:59.383472 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:59.383380    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:59.676376 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:55:59.676341    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s"]
Nov 04 01:55:59.680314 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:55:59.680291    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f437497_7f45_4df4_a309_7bdd9bb697c4.slice/crio-b34c9aeff5f5fb2fbedbd42343354608fc7ed843b3931245117e496b277cc009 WatchSource:0}: Error finding container b34c9aeff5f5fb2fbedbd42343354608fc7ed843b3931245117e496b277cc009: Status 404 returned error can't find the container with id b34c9aeff5f5fb2fbedbd42343354608fc7ed843b3931245117e496b277cc009
Nov 04 01:56:00.423482 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.423429    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s" event={"ID":"0f437497-7f45-4df4-a309-7bdd9bb697c4","Type":"ContainerStarted","Data":"b34c9aeff5f5fb2fbedbd42343354608fc7ed843b3931245117e496b277cc009"}
Nov 04 01:56:00.531932 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.531898    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-576c4d8f8-8zv54"]
Nov 04 01:56:00.534804 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.534779    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.587392 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.587358    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 04 01:56:00.587575 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.587419    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pv8vt\""
Nov 04 01:56:00.628056 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.628024    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-oauth-serving-cert\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.628220 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.628080    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-oauth-config\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.628220 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.628178    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-config\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.628220 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.628216    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-service-ca\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.628347 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.628234    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56lr6\" (UniqueName: \"kubernetes.io/projected/4f781b09-233f-4bdd-9024-e52ec52b31b3-kube-api-access-56lr6\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.628347 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.628297    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-serving-cert\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.628347 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.628329    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f781b09-233f-4bdd-9024-e52ec52b31b3-tmp\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.637346 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.637315    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 04 01:56:00.637346 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.637322    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 04 01:56:00.637561 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.637378    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 04 01:56:00.637561 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.637388    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 04 01:56:00.638182 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.638160    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-576c4d8f8-8zv54"]
Nov 04 01:56:00.729281 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.729197    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-service-ca\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.729281 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.729234    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56lr6\" (UniqueName: \"kubernetes.io/projected/4f781b09-233f-4bdd-9024-e52ec52b31b3-kube-api-access-56lr6\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.729281 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.729259    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-serving-cert\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.729281 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.729281    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f781b09-233f-4bdd-9024-e52ec52b31b3-tmp\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.729617 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.729316    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-oauth-serving-cert\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.729617 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.729342    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-oauth-config\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.729617 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.729362    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-config\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.729918 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.729891    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f781b09-233f-4bdd-9024-e52ec52b31b3-tmp\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.729992 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.729933    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-config\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.729992 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.729976    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-service-ca\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.730396 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.730377    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-oauth-serving-cert\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.732727 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.732701    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-serving-cert\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.733462 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.733435    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-oauth-config\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.790677 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.790643    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56lr6\" (UniqueName: \"kubernetes.io/projected/4f781b09-233f-4bdd-9024-e52ec52b31b3-kube-api-access-56lr6\") pod \"console-576c4d8f8-8zv54\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") " pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:00.847685 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:00.847439    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:01.003041 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:01.003006    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:01.207365 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:01.207326    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:01.224241 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:01.224210    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-576c4d8f8-8zv54"]
Nov 04 01:56:01.227900 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:56:01.227867    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f781b09_233f_4bdd_9024_e52ec52b31b3.slice/crio-35498a89a70307a7f66ec076f766a30c34c89af0cd774c61ecdb5bced5e810e6 WatchSource:0}: Error finding container 35498a89a70307a7f66ec076f766a30c34c89af0cd774c61ecdb5bced5e810e6: Status 404 returned error can't find the container with id 35498a89a70307a7f66ec076f766a30c34c89af0cd774c61ecdb5bced5e810e6
Nov 04 01:56:01.423653 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:01.423627    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:01.427246 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:01.427222    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c4d8f8-8zv54" event={"ID":"4f781b09-233f-4bdd-9024-e52ec52b31b3","Type":"ContainerStarted","Data":"35498a89a70307a7f66ec076f766a30c34c89af0cd774c61ecdb5bced5e810e6"}
Nov 04 01:56:02.318275 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:02.318245    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:02.432839 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:02.432797    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s" event={"ID":"0f437497-7f45-4df4-a309-7bdd9bb697c4","Type":"ContainerStarted","Data":"73e7fedea0646e8a40017e9e40b20a202182ee2ff2338fc5621ef538bf09df9d"}
Nov 04 01:56:02.433355 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:02.433062    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s"
Nov 04 01:56:02.439202 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:02.439179    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s"
Nov 04 01:56:02.587258 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:02.587148    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:02.664310 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:02.663977    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2526s" podStartSLOduration=2.929229817 podStartE2EDuration="4.663958351s" totalImagesPullingTime="1.734728534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:02.548266689 +0000 UTC m=+24.353723319" watchObservedRunningTime="2026-11-04 01:56:02.663958351 +0000 UTC m=+24.469414983"
Nov 04 01:56:02.968213 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:02.968118    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:03.167845 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:03.167809    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:03.373445 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:03.373409    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:03.579370 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:03.577592    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:03.908027 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:03.907993    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:05.446791 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:05.446718    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c4d8f8-8zv54" event={"ID":"4f781b09-233f-4bdd-9024-e52ec52b31b3","Type":"ContainerStarted","Data":"1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954"}
Nov 04 01:56:05.564265 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:05.564204    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-576c4d8f8-8zv54" podStartSLOduration=1.941540726 podStartE2EDuration="5.56418652s" totalImagesPullingTime="3.622645794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:05.563572035 +0000 UTC m=+27.369028666" watchObservedRunningTime="2026-11-04 01:56:05.56418652 +0000 UTC m=+27.369643151"
Nov 04 01:56:06.155677 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:06.155632    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:06.356620 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:06.356582    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:07.667641 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:07.667608    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jnspf"
Nov 04 01:56:08.796109 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:08.796060    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:09.008804 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:09.008769    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:09.225930 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:09.225856    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:09.525287 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:09.525246    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:09.958938 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:09.958829    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:10.161075 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.160930    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:10.202950 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.202917    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"]
Nov 04 01:56:10.236204 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.236175    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.271397 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.271317    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"]
Nov 04 01:56:10.311475 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.311444    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.311654 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.311537    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.311654 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.311613    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.311654 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.311638    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc56m\" (UniqueName: \"kubernetes.io/projected/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-kube-api-access-rc56m\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.319659 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.319605    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 01:56:10.319659 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.319603    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 01:56:10.319830 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.319716    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 01:56:10.342155 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.342128    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"]
Nov 04 01:56:10.360573 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.360548    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.369408 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.369383    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 04 01:56:10.369408 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.369401    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 04 01:56:10.369624 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.369433    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-vl964\""
Nov 04 01:56:10.370276 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.370253    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9dlvz"]
Nov 04 01:56:10.395112 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.392759    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.395112 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.394130    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:10.412281 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.412245    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.412410 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.412332    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.412410 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.412367    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.412410 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.412393    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc56m\" (UniqueName: \"kubernetes.io/projected/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-kube-api-access-rc56m\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.413238 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.413215    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.415136 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.415110    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.415274 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.415251    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.437499 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.437473    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"]
Nov 04 01:56:10.471484 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.471433    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 01:56:10.471647 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.471490    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 01:56:10.473660 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.473631    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc56m\" (UniqueName: \"kubernetes.io/projected/3d10c3f0-fede-4fec-bdbc-bfdfdf4788df-kube-api-access-rc56m\") pod \"openshift-state-metrics-7bd7489957-zbq9m\" (UID: \"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.493188 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.493103    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 01:56:10.493188 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.493112    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hccdr\""
Nov 04 01:56:10.493188 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.493114    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 01:56:10.493188 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.493182    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ftsng\""
Nov 04 01:56:10.493461 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.493240    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 01:56:10.493461 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.493244    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 01:56:10.512931 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.512899    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85fgb\" (UniqueName: \"kubernetes.io/projected/c6137457-318e-414f-9351-7ccf6ed64333-kube-api-access-85fgb\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.513085 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.512942    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a557337-43bd-42f2-809c-1ded60fbff48-sys\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.513085 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513024    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-textfile\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.513085 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513056    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gv4b\" (UniqueName: \"kubernetes.io/projected/0a557337-43bd-42f2-809c-1ded60fbff48-kube-api-access-5gv4b\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.513242 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513086    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6137457-318e-414f-9351-7ccf6ed64333-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.513242 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513189    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.513242 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513220    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-accelerators-collector-config\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.513693 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513249    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-tls\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.513693 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513327    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a557337-43bd-42f2-809c-1ded60fbff48-root\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.513693 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513369    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a557337-43bd-42f2-809c-1ded60fbff48-metrics-client-ca\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.513693 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513394    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6137457-318e-414f-9351-7ccf6ed64333-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.513693 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513444    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c6137457-318e-414f-9351-7ccf6ed64333-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.513693 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513491    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6137457-318e-414f-9351-7ccf6ed64333-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.513693 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513536    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c6137457-318e-414f-9351-7ccf6ed64333-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.513693 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.513569    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-wtmp\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.586960 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.586929    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"
Nov 04 01:56:10.599616 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.599582    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:10.613972 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.613940    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gv4b\" (UniqueName: \"kubernetes.io/projected/0a557337-43bd-42f2-809c-1ded60fbff48-kube-api-access-5gv4b\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614097 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.613994    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6137457-318e-414f-9351-7ccf6ed64333-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.614097 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614029    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614097 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614062    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-accelerators-collector-config\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614097 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614092    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-tls\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614113    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a557337-43bd-42f2-809c-1ded60fbff48-root\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614138    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a557337-43bd-42f2-809c-1ded60fbff48-metrics-client-ca\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614161    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6137457-318e-414f-9351-7ccf6ed64333-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.614369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614185    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c6137457-318e-414f-9351-7ccf6ed64333-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.614369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614220    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6137457-318e-414f-9351-7ccf6ed64333-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.614369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614247    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c6137457-318e-414f-9351-7ccf6ed64333-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.614369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614280    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-wtmp\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614302    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85fgb\" (UniqueName: \"kubernetes.io/projected/c6137457-318e-414f-9351-7ccf6ed64333-kube-api-access-85fgb\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.614369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614316    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a557337-43bd-42f2-809c-1ded60fbff48-sys\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614369 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614333    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-textfile\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614899 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614626    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a557337-43bd-42f2-809c-1ded60fbff48-root\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614899 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614647    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-textfile\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614899 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614784    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-wtmp\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.614899 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.614897    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c6137457-318e-414f-9351-7ccf6ed64333-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.615107 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.615093    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a557337-43bd-42f2-809c-1ded60fbff48-metrics-client-ca\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.615165 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.615134    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a557337-43bd-42f2-809c-1ded60fbff48-sys\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.615660 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.615633    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6137457-318e-414f-9351-7ccf6ed64333-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.615764 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.615721    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-accelerators-collector-config\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.615818 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.615760    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c6137457-318e-414f-9351-7ccf6ed64333-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.618237 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.617880    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.618237 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.618143    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6137457-318e-414f-9351-7ccf6ed64333-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.618237 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.618188    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6137457-318e-414f-9351-7ccf6ed64333-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.619646 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.619604    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a557337-43bd-42f2-809c-1ded60fbff48-node-exporter-tls\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.678176 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.677315    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85fgb\" (UniqueName: \"kubernetes.io/projected/c6137457-318e-414f-9351-7ccf6ed64333-kube-api-access-85fgb\") pod \"kube-state-metrics-5dc8d7d89b-7zdv8\" (UID: \"c6137457-318e-414f-9351-7ccf6ed64333\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.680168 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.679640    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gv4b\" (UniqueName: \"kubernetes.io/projected/0a557337-43bd-42f2-809c-1ded60fbff48-kube-api-access-5gv4b\") pod \"node-exporter-9dlvz\" (UID: \"0a557337-43bd-42f2-809c-1ded60fbff48\") " pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.713561 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.713528    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"
Nov 04 01:56:10.756075 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.756045    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9dlvz"
Nov 04 01:56:10.805578 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.805545    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:10.848066 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.848025    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:10.848242 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.848099    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:10.853591 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:10.853158    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:11.028144 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:11.028118    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m"]
Nov 04 01:56:11.030126 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:56:11.030091    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d10c3f0_fede_4fec_bdbc_bfdfdf4788df.slice/crio-c2e4b9f0b91fb09d3a286ff381feb9d221aed3e8a29d06025de00d4256bb1d3f WatchSource:0}: Error finding container c2e4b9f0b91fb09d3a286ff381feb9d221aed3e8a29d06025de00d4256bb1d3f: Status 404 returned error can't find the container with id c2e4b9f0b91fb09d3a286ff381feb9d221aed3e8a29d06025de00d4256bb1d3f
Nov 04 01:56:11.035300 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:11.035275    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:11.130436 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:11.130409    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8"]
Nov 04 01:56:11.140341 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:56:11.140300    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6137457_318e_414f_9351_7ccf6ed64333.slice/crio-e5661745328c3ef33efec37d89abad5be489ce68062fe8ec7c7a3eb661a2360c WatchSource:0}: Error finding container e5661745328c3ef33efec37d89abad5be489ce68062fe8ec7c7a3eb661a2360c: Status 404 returned error can't find the container with id e5661745328c3ef33efec37d89abad5be489ce68062fe8ec7c7a3eb661a2360c
Nov 04 01:56:11.470613 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:11.470520    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8" event={"ID":"c6137457-318e-414f-9351-7ccf6ed64333","Type":"ContainerStarted","Data":"e5661745328c3ef33efec37d89abad5be489ce68062fe8ec7c7a3eb661a2360c"}
Nov 04 01:56:11.472743 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:11.472707    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m" event={"ID":"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df","Type":"ContainerStarted","Data":"f336779a5698225424841a8e3f9a4e5f317b9939685c40dbae345ea31509e1b3"}
Nov 04 01:56:11.472897 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:11.472779    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m" event={"ID":"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df","Type":"ContainerStarted","Data":"08d9cbfa20e43a971e7ef19fd0c66bc91d8ac46f184b24490a95c2cd81ed58fc"}
Nov 04 01:56:11.472897 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:11.472812    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m" event={"ID":"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df","Type":"ContainerStarted","Data":"c2e4b9f0b91fb09d3a286ff381feb9d221aed3e8a29d06025de00d4256bb1d3f"}
Nov 04 01:56:11.474065 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:11.474026    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9dlvz" event={"ID":"0a557337-43bd-42f2-809c-1ded60fbff48","Type":"ContainerStarted","Data":"1cf6de699042a248328cd1ed4c29b7801c9d100c91d4d4f4cd7ee013730bb105"}
Nov 04 01:56:11.478661 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:11.478639    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:11.751386 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:11.751312    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:12.086542 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.086442    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:12.621703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.621671    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 01:56:12.637829 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.637796    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.637972 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.637941    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-f6795c6b4-6njmw"]
Nov 04 01:56:12.643917 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.643887    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:12.646002 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.645902    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.691410 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.691382    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 01:56:12.692781 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.692757    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 01:56:12.692914 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.692832    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 01:56:12.692982 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.692893    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 01:56:12.693079 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.693055    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 01:56:12.699653 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.699619    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t678k\""
Nov 04 01:56:12.699765 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.699705    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 01:56:12.699765 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.699719    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 01:56:12.699765 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.699706    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 01:56:12.733061 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733022    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733229 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733077    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-config\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.733229 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733108    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-config-volume\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733229 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733129    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-trusted-ca-bundle\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.733229 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733159    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-serving-cert\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.733229 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733186    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-config-out\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733229 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733200    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733229 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733220    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733234    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-oauth-serving-cert\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733253    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96f6p\" (UniqueName: \"kubernetes.io/projected/6fdfc9f7-60e3-43f3-b85d-9adf12316697-kube-api-access-96f6p\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733279    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733301    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733336    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733354    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-service-ca\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733372    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-web-config\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733393    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mv6c\" (UniqueName: \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-kube-api-access-6mv6c\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733416    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-oauth-config\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733450    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733474    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733500    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.733614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.733547    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fdfc9f7-60e3-43f3-b85d-9adf12316697-tmp\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.799407 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.799323    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 04 01:56:12.799407 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.799381    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 01:56:12.802015 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.801964    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f6795c6b4-6njmw"]
Nov 04 01:56:12.802141 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.802030    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 01:56:12.834969 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.834935    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835127 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.834982    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-service-ca\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.835127 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835007    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-web-config\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835127 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835032    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mv6c\" (UniqueName: \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-kube-api-access-6mv6c\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835127 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835056    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-oauth-config\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.835127 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835084    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835127 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835106    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835377 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835129    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835377 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835152    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fdfc9f7-60e3-43f3-b85d-9adf12316697-tmp\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.835377 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835203    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835377 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835229    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-config\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.835377 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835254    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-config-volume\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835377 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835276    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-trusted-ca-bundle\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.835377 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835305    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-serving-cert\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.835377 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835347    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-config-out\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835377 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835373    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835738 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835396    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835738 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835430    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-oauth-serving-cert\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.835738 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835453    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96f6p\" (UniqueName: \"kubernetes.io/projected/6fdfc9f7-60e3-43f3-b85d-9adf12316697-kube-api-access-96f6p\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.835738 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835488    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835738 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835534    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.835950 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.835760    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-service-ca\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.836270 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.836243    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.836481 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.836409    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.838140 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.837351    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-oauth-serving-cert\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.838140 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.837841    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-config\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.838140 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.837959    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.838140 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.838112    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fdfc9f7-60e3-43f3-b85d-9adf12316697-tmp\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.839095 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.838467    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.839260 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.839195    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-trusted-ca-bundle\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.840813 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.840768    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-oauth-config\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.841052 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.841024    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.841157 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.841135    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.841211 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.841139    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.841626 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.841601    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.842410 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.842390    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-config-out\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.843023 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.843002    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-config-volume\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.843691 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.843670    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.844953 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.844932    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-web-config\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.845964 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.845939    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-serving-cert\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.903414 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.902829    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96f6p\" (UniqueName: \"kubernetes.io/projected/6fdfc9f7-60e3-43f3-b85d-9adf12316697-kube-api-access-96f6p\") pod \"console-f6795c6b4-6njmw\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") " pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:12.904705 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.903816    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mv6c\" (UniqueName: \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-kube-api-access-6mv6c\") pod \"alertmanager-main-0\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:12.935048 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.935016    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"]
Nov 04 01:56:12.949913 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.949886    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:12.979713 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:12.979683    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:13.001820 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.001790    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"]
Nov 04 01:56:13.005424 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.004872    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a82174uelh0lb\""
Nov 04 01:56:13.005424 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.004962    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 04 01:56:13.005424 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.005068    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-sc6ll\""
Nov 04 01:56:13.005424 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.005128    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 04 01:56:13.005424 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.005207    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 04 01:56:13.005424 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.005265    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 04 01:56:13.005424 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.005356    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 04 01:56:13.013050 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.012866    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:56:13.039642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.039093    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afeff4a9-d88a-4cd6-9127-7531da99f4af-metrics-client-ca\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.039642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.039204    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.039642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.039240    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.039642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.039266    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afeff4a9-d88a-4cd6-9127-7531da99f4af-tmp\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.039642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.039298    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.039642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.039342    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tz6p\" (UniqueName: \"kubernetes.io/projected/afeff4a9-d88a-4cd6-9127-7531da99f4af-kube-api-access-2tz6p\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.039642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.039375    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-tls\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.039642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.039399    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.039642 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.039432    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-grpc-tls\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.140782 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.140744    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.141116 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.140802    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afeff4a9-d88a-4cd6-9127-7531da99f4af-tmp\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.141116 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.140833    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.141116 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.140884    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tz6p\" (UniqueName: \"kubernetes.io/projected/afeff4a9-d88a-4cd6-9127-7531da99f4af-kube-api-access-2tz6p\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.141116 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.140920    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-tls\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.141116 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.140944    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.141116 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.140973    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-grpc-tls\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.141116 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.140997    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afeff4a9-d88a-4cd6-9127-7531da99f4af-metrics-client-ca\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.141116 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.141103    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.141417 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.141275    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afeff4a9-d88a-4cd6-9127-7531da99f4af-tmp\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.144697 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.144667    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afeff4a9-d88a-4cd6-9127-7531da99f4af-metrics-client-ca\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.148492 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.147639    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.148492 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.148448    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.149365 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.149307    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.149920 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.149872    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.152302 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.152279    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-grpc-tls\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.152725 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.152702    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/afeff4a9-d88a-4cd6-9127-7531da99f4af-secret-thanos-querier-tls\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.210346 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.210314    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tz6p\" (UniqueName: \"kubernetes.io/projected/afeff4a9-d88a-4cd6-9127-7531da99f4af-kube-api-access-2tz6p\") pod \"thanos-querier-6fbfb964d-m4zw5\" (UID: \"afeff4a9-d88a-4cd6-9127-7531da99f4af\") " pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.335231 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.333799    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:13.434189 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.434153    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f6795c6b4-6njmw"]
Nov 04 01:56:13.452099 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:56:13.439028    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fdfc9f7_60e3_43f3_b85d_9adf12316697.slice/crio-f19e0b065bd288ad09a399a891e233489e2c594f40d311c25bbc402f1ac03bef WatchSource:0}: Error finding container f19e0b065bd288ad09a399a891e233489e2c594f40d311c25bbc402f1ac03bef: Status 404 returned error can't find the container with id f19e0b065bd288ad09a399a891e233489e2c594f40d311c25bbc402f1ac03bef
Nov 04 01:56:13.493818 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.493786    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 01:56:13.529798 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.529766    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-dblsg" event={"ID":"49445752-7958-49b2-afb3-2c04bc41ec3a","Type":"ContainerStarted","Data":"01125c0feb80f326f280b4faa14ffe3da105b0d4c74ac38d785b6f24d7389bb6"}
Nov 04 01:56:13.529916 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.529829    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-dblsg"
Nov 04 01:56:13.537035 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.537001    4512 generic.go:356] "Generic (PLEG): container finished" podID="0a557337-43bd-42f2-809c-1ded60fbff48" containerID="01058c8d8f7c3149498b7be6ce9f15b1d2e7357988dab643c28eac628f49207a" exitCode=0
Nov 04 01:56:13.537188 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.537090    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9dlvz" event={"ID":"0a557337-43bd-42f2-809c-1ded60fbff48","Type":"ContainerDied","Data":"01058c8d8f7c3149498b7be6ce9f15b1d2e7357988dab643c28eac628f49207a"}
Nov 04 01:56:13.556922 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.540975    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-dblsg"
Nov 04 01:56:13.556922 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.543259    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6795c6b4-6njmw" event={"ID":"6fdfc9f7-60e3-43f3-b85d-9adf12316697","Type":"ContainerStarted","Data":"f19e0b065bd288ad09a399a891e233489e2c594f40d311c25bbc402f1ac03bef"}
Nov 04 01:56:13.556922 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.545923    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8" event={"ID":"c6137457-318e-414f-9351-7ccf6ed64333","Type":"ContainerStarted","Data":"3f0449dcd2007fb58f883c3b603614afd6e16a7c15dfe9b27343db63060ef648"}
Nov 04 01:56:13.556922 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.546007    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8" event={"ID":"c6137457-318e-414f-9351-7ccf6ed64333","Type":"ContainerStarted","Data":"61ff9b399ccd194205849162e060fa453812a6cc9c62ca0d51f78341f3467fe7"}
Nov 04 01:56:13.647573 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.645880    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-dblsg" podStartSLOduration=2.294438703 podStartE2EDuration="21.645862742s" totalImagesPullingTime="19.351424039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:13.644975207 +0000 UTC m=+35.450431851" watchObservedRunningTime="2026-11-04 01:56:13.645862742 +0000 UTC m=+35.451319376"
Nov 04 01:56:13.770668 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:13.770632    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"]
Nov 04 01:56:14.556548 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.555643    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m" event={"ID":"3d10c3f0-fede-4fec-bdbc-bfdfdf4788df","Type":"ContainerStarted","Data":"71bf800badce2dbb6cb1fc03b4eac21b5bfbdb3a71bf77668ca2de0935bdc354"}
Nov 04 01:56:14.559674 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.559590    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerStarted","Data":"503a7bd7f691b03600f8f81846db9d970373f2f0d579ba4349da2114fd4987e8"}
Nov 04 01:56:14.563319 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.563295    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5" event={"ID":"afeff4a9-d88a-4cd6-9127-7531da99f4af","Type":"ContainerStarted","Data":"116362021b570756eb10a974622c05d4760a0bd9a2711e3addd04be830fb69b6"}
Nov 04 01:56:14.568076 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.567586    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9dlvz" event={"ID":"0a557337-43bd-42f2-809c-1ded60fbff48","Type":"ContainerStarted","Data":"7d63931c9b33370d27f9c4d7dbe8afcdb1800784b4362fe8d24f9abd5b662186"}
Nov 04 01:56:14.568076 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.568059    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9dlvz" event={"ID":"0a557337-43bd-42f2-809c-1ded60fbff48","Type":"ContainerStarted","Data":"05f4f5029f605fed0c47f26e0a5384694592dcc0650fbe7c735f18d18c22a910"}
Nov 04 01:56:14.570785 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.570740    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6795c6b4-6njmw" event={"ID":"6fdfc9f7-60e3-43f3-b85d-9adf12316697","Type":"ContainerStarted","Data":"3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b"}
Nov 04 01:56:14.574664 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.574623    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8" event={"ID":"c6137457-318e-414f-9351-7ccf6ed64333","Type":"ContainerStarted","Data":"6dd04d83c6ee9ea29b33874e70a8fc5186ed385c3b638c1dd8dbe3d8513b141e"}
Nov 04 01:56:14.653304 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.653255    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:14.798665 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.798465    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-zbq9m" podStartSLOduration=1.857706724 podStartE2EDuration="4.798446004s" totalImagesPullingTime="2.94073928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:14.677334669 +0000 UTC m=+36.482791303" watchObservedRunningTime="2026-11-04 01:56:14.798446004 +0000 UTC m=+36.603902636"
Nov 04 01:56:14.799294 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.799258    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7zdv8" podStartSLOduration=2.744176448 podStartE2EDuration="4.799249548s" totalImagesPullingTime="2.0550731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:14.79784438 +0000 UTC m=+36.603301011" watchObservedRunningTime="2026-11-04 01:56:14.799249548 +0000 UTC m=+36.604706179"
Nov 04 01:56:14.852632 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.852263    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:14.974165 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.973260    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9dlvz" podStartSLOduration=2.727251766 podStartE2EDuration="4.973238543s" totalImagesPullingTime="2.14082331s" totalInitContainerRuntime="105.163467ms" isStatefulPod=true podCreationTimestamp="2026-11-04 01:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:14.921220497 +0000 UTC m=+36.726677126" watchObservedRunningTime="2026-11-04 01:56:14.973238543 +0000 UTC m=+36.778695175"
Nov 04 01:56:14.974165 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.973708    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-zp926"]
Nov 04 01:56:14.993348 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:14.992752    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:15.043254 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.043220    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-zp926"]
Nov 04 01:56:15.063007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.062849    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fljhg\" (UniqueName: \"kubernetes.io/projected/db8099f3-2446-4394-907b-5c30917a084f-kube-api-access-fljhg\") pod \"monitoring-plugin-77f4f76594-zp926\" (UID: \"db8099f3-2446-4394-907b-5c30917a084f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:15.063007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.062899    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db8099f3-2446-4394-907b-5c30917a084f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-zp926\" (UID: \"db8099f3-2446-4394-907b-5c30917a084f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:15.080217 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.080173    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:15.115756 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.115529    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 04 01:56:15.115756 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.115687    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rngcs\""
Nov 04 01:56:15.167268 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.166035    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fljhg\" (UniqueName: \"kubernetes.io/projected/db8099f3-2446-4394-907b-5c30917a084f-kube-api-access-fljhg\") pod \"monitoring-plugin-77f4f76594-zp926\" (UID: \"db8099f3-2446-4394-907b-5c30917a084f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:15.167268 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.166086    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db8099f3-2446-4394-907b-5c30917a084f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-zp926\" (UID: \"db8099f3-2446-4394-907b-5c30917a084f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:15.167268 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:56:15.166270    4512 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Nov 04 01:56:15.167268 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:56:15.166331    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db8099f3-2446-4394-907b-5c30917a084f-monitoring-plugin-cert podName:db8099f3-2446-4394-907b-5c30917a084f nodeName:}" failed. No retries permitted until 2026-11-04 01:56:15.666309464 +0000 UTC m=+37.471766093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/db8099f3-2446-4394-907b-5c30917a084f-monitoring-plugin-cert") pod "monitoring-plugin-77f4f76594-zp926" (UID: "db8099f3-2446-4394-907b-5c30917a084f") : secret "monitoring-plugin-cert" not found
Nov 04 01:56:15.167268 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.166905    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-f6795c6b4-6njmw" podStartSLOduration=3.166886665 podStartE2EDuration="3.166886665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:15.049366823 +0000 UTC m=+36.854823455" watchObservedRunningTime="2026-11-04 01:56:15.166886665 +0000 UTC m=+36.972343299"
Nov 04 01:56:15.189285 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.187344    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5584fc7458-wcht4"]
Nov 04 01:56:15.202581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.202553    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.228163 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.228133    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fljhg\" (UniqueName: \"kubernetes.io/projected/db8099f3-2446-4394-907b-5c30917a084f-kube-api-access-fljhg\") pod \"monitoring-plugin-77f4f76594-zp926\" (UID: \"db8099f3-2446-4394-907b-5c30917a084f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:15.250679 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.249862    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5584fc7458-wcht4"]
Nov 04 01:56:15.256566 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.256529    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 01:56:15.256723 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.256646    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 04 01:56:15.267926 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.267890    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-client-ca-bundle\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.268065 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.267942    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-audit-log\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.268065 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.268017    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64vhh\" (UniqueName: \"kubernetes.io/projected/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-kube-api-access-64vhh\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.268065 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.268054    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-secret-metrics-server-client-certs\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.268230 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.268087    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-metrics-server-audit-profiles\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.268230 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.268133    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.268230 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.268170    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-secret-metrics-server-tls\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.291681 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.291440    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:15.325836 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.325794    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 04 01:56:15.326140 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.326019    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-kd7bugpphe95\""
Nov 04 01:56:15.326140 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.326096    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kkqk8\""
Nov 04 01:56:15.326423 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.326387    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 04 01:56:15.369689 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.369562    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-client-ca-bundle\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.369689 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.369622    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-audit-log\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.369913 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.369710    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64vhh\" (UniqueName: \"kubernetes.io/projected/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-kube-api-access-64vhh\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.369913 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.369744    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-secret-metrics-server-client-certs\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.369913 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.369764    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-metrics-server-audit-profiles\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.369913 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.369792    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.369913 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.369817    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-secret-metrics-server-tls\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.370909 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.370855    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-audit-log\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.372077 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.372040    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-metrics-server-audit-profiles\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.373175 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.373145    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.374729 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.374706    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-secret-metrics-server-tls\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.375394 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.375368    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-client-ca-bundle\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.376734 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.376708    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-secret-metrics-server-client-certs\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.435062 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.435031    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64vhh\" (UniqueName: \"kubernetes.io/projected/a71b41b2-6ae5-47cb-b2e6-6df4a70f2919-kube-api-access-64vhh\") pod \"metrics-server-5584fc7458-wcht4\" (UID: \"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919\") " pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.489494 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.489454    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:15.527564 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.527527    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:15.673361 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.673287    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db8099f3-2446-4394-907b-5c30917a084f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-zp926\" (UID: \"db8099f3-2446-4394-907b-5c30917a084f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:15.680753 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.680725    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/db8099f3-2446-4394-907b-5c30917a084f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-zp926\" (UID: \"db8099f3-2446-4394-907b-5c30917a084f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:15.691927 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.691883    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:15.924310 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.924047    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:15.987035 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:15.987001    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5584fc7458-wcht4"]
Nov 04 01:56:15.998683 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:56:15.997220    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71b41b2_6ae5_47cb_b2e6_6df4a70f2919.slice/crio-84552c0a2d5793d1d857499c46d21f802b03387b71161cf5d41b4575429db75e WatchSource:0}: Error finding container 84552c0a2d5793d1d857499c46d21f802b03387b71161cf5d41b4575429db75e: Status 404 returned error can't find the container with id 84552c0a2d5793d1d857499c46d21f802b03387b71161cf5d41b4575429db75e
Nov 04 01:56:16.318902 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:16.318866    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-zp926"]
Nov 04 01:56:16.322333 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:56:16.322303    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb8099f3_2446_4394_907b_5c30917a084f.slice/crio-715f61b14f5f6bb28e827faf42b0f322df2af2d7663e6e5792042240c1bf6f82 WatchSource:0}: Error finding container 715f61b14f5f6bb28e827faf42b0f322df2af2d7663e6e5792042240c1bf6f82: Status 404 returned error can't find the container with id 715f61b14f5f6bb28e827faf42b0f322df2af2d7663e6e5792042240c1bf6f82
Nov 04 01:56:16.590028 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:16.589725    4512 generic.go:356] "Generic (PLEG): container finished" podID="3da135c1-ccf8-4102-b3fb-59768928c672" containerID="650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b" exitCode=0
Nov 04 01:56:16.590405 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:16.590307    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerDied","Data":"650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b"}
Nov 04 01:56:16.593696 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:16.593264    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5584fc7458-wcht4" event={"ID":"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919","Type":"ContainerStarted","Data":"84552c0a2d5793d1d857499c46d21f802b03387b71161cf5d41b4575429db75e"}
Nov 04 01:56:16.596853 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:16.596804    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926" event={"ID":"db8099f3-2446-4394-907b-5c30917a084f","Type":"ContainerStarted","Data":"715f61b14f5f6bb28e827faf42b0f322df2af2d7663e6e5792042240c1bf6f82"}
Nov 04 01:56:16.980193 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:16.980039    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:17.113375 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.113336    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f6795c6b4-6njmw"]
Nov 04 01:56:17.141217 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.141186    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-8c458698d-h589f"]
Nov 04 01:56:17.164406 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.164018    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.193129 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.193088    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:17.207271 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.207201    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8c458698d-h589f"]
Nov 04 01:56:17.291856 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.291349    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-config\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.291856 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.291399    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-serving-cert\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.291856 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.291440    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-oauth-serving-cert\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.291856 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.291480    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48rgr\" (UniqueName: \"kubernetes.io/projected/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-kube-api-access-48rgr\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.291856 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.291530    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-oauth-config\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.291856 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.291580    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-service-ca\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.291856 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.291611    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-tmp\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.291856 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.291641    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-trusted-ca-bundle\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.393158 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.393117    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-trusted-ca-bundle\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.393805 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.393182    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-config\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.393805 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.393357    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-serving-cert\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.394017 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.393981    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-oauth-serving-cert\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.394283 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.394035    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48rgr\" (UniqueName: \"kubernetes.io/projected/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-kube-api-access-48rgr\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.394283 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.394066    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-oauth-config\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.394283 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.394127    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-service-ca\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.394283 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.394165    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-tmp\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.394552 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.394531    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-tmp\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.395025 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.394989    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-config\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.395192 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.395173    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-service-ca\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.395353 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.395328    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-trusted-ca-bundle\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.396823 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.396598    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-oauth-serving-cert\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.400465 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.400439    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-oauth-config\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.402870 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.402835    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-serving-cert\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.459306 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.459242    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48rgr\" (UniqueName: \"kubernetes.io/projected/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-kube-api-access-48rgr\") pod \"console-8c458698d-h589f\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") " pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.493909 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.493616    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:17.934229 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:17.931799    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8c458698d-h589f"]
Nov 04 01:56:17.934229 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:56:17.933225    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fffa60b_7ee8_4cdc_a8b8_c09ead0f82a7.slice/crio-de7587ba0a36ce6f931b03e33e679c2d36e22be932b6f553338f122ebc42da80 WatchSource:0}: Error finding container de7587ba0a36ce6f931b03e33e679c2d36e22be932b6f553338f122ebc42da80: Status 404 returned error can't find the container with id de7587ba0a36ce6f931b03e33e679c2d36e22be932b6f553338f122ebc42da80
Nov 04 01:56:18.611326 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:18.611259    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c458698d-h589f" event={"ID":"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7","Type":"ContainerStarted","Data":"6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf"}
Nov 04 01:56:18.611836 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:18.611345    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c458698d-h589f" event={"ID":"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7","Type":"ContainerStarted","Data":"de7587ba0a36ce6f931b03e33e679c2d36e22be932b6f553338f122ebc42da80"}
Nov 04 01:56:18.728878 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:18.728800    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-8c458698d-h589f" podStartSLOduration=1.728786414 podStartE2EDuration="1.728786414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:18.728243149 +0000 UTC m=+40.533699782" watchObservedRunningTime="2026-11-04 01:56:18.728786414 +0000 UTC m=+40.534243044"
Nov 04 01:56:19.564124 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:19.564069    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:19.799423 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:19.799392    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:20.622727 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.622696    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926" event={"ID":"db8099f3-2446-4394-907b-5c30917a084f","Type":"ContainerStarted","Data":"d01d13c5fcac231a4acecd2ab9ca641be67bf5f2560459de814e36e9e8136610"}
Nov 04 01:56:20.623364 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.623345    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:20.626483 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.626013    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerStarted","Data":"1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f"}
Nov 04 01:56:20.626483 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.626106    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerStarted","Data":"844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09"}
Nov 04 01:56:20.628905 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.628890    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926"
Nov 04 01:56:20.629840 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.629811    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5" event={"ID":"afeff4a9-d88a-4cd6-9127-7531da99f4af","Type":"ContainerStarted","Data":"a87097bc08f91e3de9a9c008ef1291ecb052f6746a6c283625e0acc1cbc42100"}
Nov 04 01:56:20.629925 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.629881    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5" event={"ID":"afeff4a9-d88a-4cd6-9127-7531da99f4af","Type":"ContainerStarted","Data":"91e7e2363743e5a25f893dbf905bbf23189bdb60ef32a7a3cfc2cadab48ed3fb"}
Nov 04 01:56:20.629925 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.629921    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5" event={"ID":"afeff4a9-d88a-4cd6-9127-7531da99f4af","Type":"ContainerStarted","Data":"3cf9ab705ccf1bb128ef30a766d16ae5d68bff7510c9aab51076e299a2128645"}
Nov 04 01:56:20.631530 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.631485    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5584fc7458-wcht4" event={"ID":"a71b41b2-6ae5-47cb-b2e6-6df4a70f2919","Type":"ContainerStarted","Data":"28553f38ba13aaa72f4df163f0d39141990bb4be176de343117c732cec74b359"}
Nov 04 01:56:20.739368 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.739315    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-zp926" podStartSLOduration=3.32800336 podStartE2EDuration="6.739301472s" totalImagesPullingTime="3.411298112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:20.737790162 +0000 UTC m=+42.543246794" watchObservedRunningTime="2026-11-04 01:56:20.739301472 +0000 UTC m=+42.544758146"
Nov 04 01:56:20.970556 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.970470    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5584fc7458-wcht4" podStartSLOduration=2.239516651 podStartE2EDuration="5.970455048s" totalImagesPullingTime="3.730938397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:20.970206993 +0000 UTC m=+42.775663626" watchObservedRunningTime="2026-11-04 01:56:20.970455048 +0000 UTC m=+42.775911679"
Nov 04 01:56:20.974677 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:20.974645    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:21.180614 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:21.180529    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:21.638418 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:21.638380    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerStarted","Data":"efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730"}
Nov 04 01:56:21.638733 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:21.638459    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerStarted","Data":"ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482"}
Nov 04 01:56:21.638733 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:21.638498    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerStarted","Data":"d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a"}
Nov 04 01:56:22.058593 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:22.058555    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:22.252844 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:22.252804    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:22.762363 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:22.762331    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:22.901115 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:22.901087    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8c458698d-h589f"]
Nov 04 01:56:22.950437 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:22.950413    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d574477cd-phdzq"]
Nov 04 01:56:22.969581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:22.969561    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:22.980408 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:22.980388    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:22.985048 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:22.985018    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:22.996743 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:22.996721    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d574477cd-phdzq"]
Nov 04 01:56:23.056809 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.056779    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-oauth-config\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.056907 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.056820    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-trusted-ca-bundle\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.056907 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.056841    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-oauth-serving-cert\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.056973 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.056951    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-serving-cert\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.057012 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.057001    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v2pj\" (UniqueName: \"kubernetes.io/projected/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-kube-api-access-4v2pj\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.057082 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.057061    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-config\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.057117 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.057108    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-service-ca\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.057146 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.057125    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-tmp\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.157735 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.157698    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v2pj\" (UniqueName: \"kubernetes.io/projected/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-kube-api-access-4v2pj\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.157735 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.157735    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-config\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.158257 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.157752    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-service-ca\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.158257 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.157877    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-tmp\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.158257 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.157999    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-oauth-config\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.158257 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.158049    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-trusted-ca-bundle\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.158257 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.158078    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-oauth-serving-cert\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.158257 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.158153    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-serving-cert\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.158257 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.158247    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-tmp\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.158649 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.158476    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-service-ca\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.158649 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.158597    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-config\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.158881 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.158862    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-trusted-ca-bundle\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.159066 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.159043    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-oauth-serving-cert\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.160641 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.160618    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-oauth-config\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.160700 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.160689    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-serving-cert\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.222234 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.222208    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v2pj\" (UniqueName: \"kubernetes.io/projected/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-kube-api-access-4v2pj\") pod \"console-5d574477cd-phdzq\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") " pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.299972 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.299893    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:23.649626 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.649530    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerStarted","Data":"c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8"}
Nov 04 01:56:23.652428 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.652402    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5" event={"ID":"afeff4a9-d88a-4cd6-9127-7531da99f4af","Type":"ContainerStarted","Data":"25857c4df75f6321cca1d897e69a8a0df0677594bc1729f15c13bad0b161c755"}
Nov 04 01:56:23.652592 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.652480    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:23.652592 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.652542    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5" event={"ID":"afeff4a9-d88a-4cd6-9127-7531da99f4af","Type":"ContainerStarted","Data":"c82abf91ff20754e79d5a9049236ff6ce87f1425ca57b54f7f065910c3081492"}
Nov 04 01:56:23.652592 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.652580    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5" event={"ID":"afeff4a9-d88a-4cd6-9127-7531da99f4af","Type":"ContainerStarted","Data":"10fbc1565f34ed80f5a9d262baef2f6f3dda8dea09475e19e089723cdb229a69"}
Nov 04 01:56:23.692162 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.692090    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d574477cd-phdzq"]
Nov 04 01:56:23.694717 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:56:23.694691    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ebd3552_420c_4bff_a1ec_16228a2d5c8a.slice/crio-6ec85d6b2ff7829ca4d7729aba7014fb5ee6b05bc2063051bbfc85381b5692dd WatchSource:0}: Error finding container 6ec85d6b2ff7829ca4d7729aba7014fb5ee6b05bc2063051bbfc85381b5692dd: Status 404 returned error can't find the container with id 6ec85d6b2ff7829ca4d7729aba7014fb5ee6b05bc2063051bbfc85381b5692dd
Nov 04 01:56:23.780004 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.779948    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.724719159 podStartE2EDuration="11.779934873s" totalImagesPullingTime="8.035959112s" totalInitContainerRuntime="19.256602ms" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:23.778036965 +0000 UTC m=+45.583493596" watchObservedRunningTime="2026-11-04 01:56:23.779934873 +0000 UTC m=+45.585391503"
Nov 04 01:56:23.902186 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:23.901712    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5" podStartSLOduration=3.213261273 podStartE2EDuration="11.901691922s" totalImagesPullingTime="8.688430649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:23.900770793 +0000 UTC m=+45.706227423" watchObservedRunningTime="2026-11-04 01:56:23.901691922 +0000 UTC m=+45.707148572"
Nov 04 01:56:24.060670 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:24.060634    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:24.291783 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:24.291751    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:24.487955 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:24.487913    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:24.658439 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:24.658342    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d574477cd-phdzq" event={"ID":"6ebd3552-420c-4bff-a1ec-16228a2d5c8a","Type":"ContainerStarted","Data":"4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f"}
Nov 04 01:56:24.658439 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:24.658403    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d574477cd-phdzq" event={"ID":"6ebd3552-420c-4bff-a1ec-16228a2d5c8a","Type":"ContainerStarted","Data":"6ec85d6b2ff7829ca4d7729aba7014fb5ee6b05bc2063051bbfc85381b5692dd"}
Nov 04 01:56:24.664623 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:24.664601    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6fbfb964d-m4zw5"
Nov 04 01:56:24.697447 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:24.697412    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:24.778202 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:24.778152    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5d574477cd-phdzq" podStartSLOduration=2.77807776 podStartE2EDuration="2.77807776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:24.777218296 +0000 UTC m=+46.582674927" watchObservedRunningTime="2026-11-04 01:56:24.77807776 +0000 UTC m=+46.583534390"
Nov 04 01:56:26.261477 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:26.261436    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:26.460901 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:26.460865    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:26.776581 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:56:26.776547    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 01:56:27.494517 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:27.494482    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:33.301042 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:33.301004    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:33.301572 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:33.301060    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:33.305490 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:33.305469    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:33.704818 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:33.704739    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:56:33.873667 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:33.873629    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-576c4d8f8-8zv54"]
Nov 04 01:56:35.528587 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:35.528552    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:35.528961 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:35.528614    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:38.785979 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:38.785935    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:39.036496 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:39.036386    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:42.137301 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.137256    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-f6795c6b4-6njmw" podUID="6fdfc9f7-60e3-43f3-b85d-9adf12316697" containerName="console" containerID="cri-o://3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b" gracePeriod=15
Nov 04 01:56:42.389094 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.389040    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:42.432668 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.432635    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-oauth-config\") pod \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") "
Nov 04 01:56:42.432830 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.432688    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fdfc9f7-60e3-43f3-b85d-9adf12316697-tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fdfc9f7-60e3-43f3-b85d-9adf12316697-tmp\") pod \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") "
Nov 04 01:56:42.432830 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.432708    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-config\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-config\") pod \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") "
Nov 04 01:56:42.432830 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.432731    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-trusted-ca-bundle\") pod \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") "
Nov 04 01:56:42.432830 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.432754    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fdfc9f7-60e3-43f3-b85d-9adf12316697-kube-api-access-96f6p\" (UniqueName: \"kubernetes.io/projected/6fdfc9f7-60e3-43f3-b85d-9adf12316697-kube-api-access-96f6p\") pod \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") "
Nov 04 01:56:42.432830 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.432777    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-oauth-serving-cert\") pod \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") "
Nov 04 01:56:42.433130 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.432831    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-service-ca\") pod \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") "
Nov 04 01:56:42.433130 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.432866    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-serving-cert\") pod \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\" (UID: \"6fdfc9f7-60e3-43f3-b85d-9adf12316697\") "
Nov 04 01:56:42.433130 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.433089    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fdfc9f7-60e3-43f3-b85d-9adf12316697-tmp" pod "6fdfc9f7-60e3-43f3-b85d-9adf12316697" (UID: "6fdfc9f7-60e3-43f3-b85d-9adf12316697"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:42.433272 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.433175    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-config" pod "6fdfc9f7-60e3-43f3-b85d-9adf12316697" (UID: "6fdfc9f7-60e3-43f3-b85d-9adf12316697"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:42.433272 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.433188    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-trusted-ca-bundle" pod "6fdfc9f7-60e3-43f3-b85d-9adf12316697" (UID: "6fdfc9f7-60e3-43f3-b85d-9adf12316697"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:42.433272 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.433194    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-oauth-serving-cert" pod "6fdfc9f7-60e3-43f3-b85d-9adf12316697" (UID: "6fdfc9f7-60e3-43f3-b85d-9adf12316697"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:42.433596 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.433446    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-service-ca" pod "6fdfc9f7-60e3-43f3-b85d-9adf12316697" (UID: "6fdfc9f7-60e3-43f3-b85d-9adf12316697"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:42.435193 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.435170    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fdfc9f7-60e3-43f3-b85d-9adf12316697-kube-api-access-96f6p" pod "6fdfc9f7-60e3-43f3-b85d-9adf12316697" (UID: "6fdfc9f7-60e3-43f3-b85d-9adf12316697"). InnerVolumeSpecName "kube-api-access-96f6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:42.435193 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.435173    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-oauth-config" pod "6fdfc9f7-60e3-43f3-b85d-9adf12316697" (UID: "6fdfc9f7-60e3-43f3-b85d-9adf12316697"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:42.435362 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.435341    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-serving-cert" pod "6fdfc9f7-60e3-43f3-b85d-9adf12316697" (UID: "6fdfc9f7-60e3-43f3-b85d-9adf12316697"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:42.533880 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.533850    4512 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-service-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.533880 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.533880    4512 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-serving-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.534060 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.533891    4512 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-oauth-config\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.534060 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.533900    4512 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fdfc9f7-60e3-43f3-b85d-9adf12316697-tmp\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.534060 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.533909    4512 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-console-config\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.534060 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.533918    4512 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-trusted-ca-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.534060 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.533926    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96f6p\" (UniqueName: \"kubernetes.io/projected/6fdfc9f7-60e3-43f3-b85d-9adf12316697-kube-api-access-96f6p\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.534060 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.533936    4512 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fdfc9f7-60e3-43f3-b85d-9adf12316697-oauth-serving-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.739359 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.739324    4512 generic.go:356] "Generic (PLEG): container finished" podID="6fdfc9f7-60e3-43f3-b85d-9adf12316697" containerID="3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b" exitCode=0
Nov 04 01:56:42.739541 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.739373    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6795c6b4-6njmw"
Nov 04 01:56:42.739541 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.739390    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6795c6b4-6njmw" event={"ID":"6fdfc9f7-60e3-43f3-b85d-9adf12316697","Type":"ContainerDied","Data":"3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b"}
Nov 04 01:56:42.739541 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.739439    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6795c6b4-6njmw" event={"ID":"6fdfc9f7-60e3-43f3-b85d-9adf12316697","Type":"ContainerDied","Data":"f19e0b065bd288ad09a399a891e233489e2c594f40d311c25bbc402f1ac03bef"}
Nov 04 01:56:42.739541 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.739465    4512 scope.go:122] "RemoveContainer" containerID="3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b"
Nov 04 01:56:42.748134 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.748113    4512 scope.go:122] "RemoveContainer" containerID="3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b"
Nov 04 01:56:42.748382 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:56:42.748361    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b\": container with ID starting with 3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b not found: ID does not exist" containerID="3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b"
Nov 04 01:56:42.748429 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.748390    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b"} err="failed to get container status \"3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b\": rpc error: code = NotFound desc = could not find container \"3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b\": container with ID starting with 3eed37967218047efe91e47e14c3f8febedd9c75e6c3fed91a2d800ac531f94b not found: ID does not exist"
Nov 04 01:56:42.902645 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.902611    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f6795c6b4-6njmw"]
Nov 04 01:56:42.952118 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:42.952084    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f6795c6b4-6njmw"]
Nov 04 01:56:44.309882 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:44.309847    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fdfc9f7-60e3-43f3-b85d-9adf12316697" path="/var/lib/kubelet/pods/6fdfc9f7-60e3-43f3-b85d-9adf12316697/volumes"
Nov 04 01:56:45.693351 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:45.693296    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:45.898872 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:45.898834    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:47.919836 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:47.919769    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-8c458698d-h589f" podUID="9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" containerName="console" containerID="cri-o://6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf" gracePeriod=15
Nov 04 01:56:48.195055 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.194988    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:48.269968 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.269934    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-oauth-config\") pod \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") "
Nov 04 01:56:48.269968 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.269972    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-oauth-serving-cert\") pod \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") "
Nov 04 01:56:48.270190 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.269994    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-kube-api-access-48rgr\" (UniqueName: \"kubernetes.io/projected/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-kube-api-access-48rgr\") pod \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") "
Nov 04 01:56:48.270190 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.270032    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-serving-cert\") pod \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") "
Nov 04 01:56:48.270190 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.270054    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-tmp\") pod \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") "
Nov 04 01:56:48.270190 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.270081    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-config\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-config\") pod \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") "
Nov 04 01:56:48.270190 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.270108    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-service-ca\") pod \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") "
Nov 04 01:56:48.270190 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.270166    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-trusted-ca-bundle\") pod \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\" (UID: \"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7\") "
Nov 04 01:56:48.270469 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.270432    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-tmp" pod "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" (UID: "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:48.270554 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.270487    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-oauth-serving-cert" pod "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" (UID: "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:48.270660 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.270638    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-config" pod "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" (UID: "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:48.270715 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.270681    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-service-ca" pod "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" (UID: "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:48.270822 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.270797    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-trusted-ca-bundle" pod "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" (UID: "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:48.272233 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.272204    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-oauth-config" pod "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" (UID: "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:48.272342 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.272319    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-kube-api-access-48rgr" pod "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" (UID: "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7"). InnerVolumeSpecName "kube-api-access-48rgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:48.272581 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.272562    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-serving-cert" pod "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" (UID: "9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:48.370766 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.370731    4512 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-oauth-config\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:48.370766 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.370759    4512 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-oauth-serving-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:48.370766 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.370769    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48rgr\" (UniqueName: \"kubernetes.io/projected/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-kube-api-access-48rgr\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:48.370984 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.370778    4512 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-serving-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:48.370984 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.370788    4512 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-tmp\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:48.370984 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.370797    4512 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-console-config\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:48.370984 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.370805    4512 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-service-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:48.370984 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.370814    4512 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7-trusted-ca-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:48.765379 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.765338    4512 generic.go:356] "Generic (PLEG): container finished" podID="9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" containerID="6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf" exitCode=0
Nov 04 01:56:48.765603 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.765405    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c458698d-h589f"
Nov 04 01:56:48.765603 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.765426    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c458698d-h589f" event={"ID":"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7","Type":"ContainerDied","Data":"6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf"}
Nov 04 01:56:48.765603 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.765531    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c458698d-h589f" event={"ID":"9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7","Type":"ContainerDied","Data":"de7587ba0a36ce6f931b03e33e679c2d36e22be932b6f553338f122ebc42da80"}
Nov 04 01:56:48.765603 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.765554    4512 scope.go:122] "RemoveContainer" containerID="6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf"
Nov 04 01:56:48.773482 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.773461    4512 scope.go:122] "RemoveContainer" containerID="6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf"
Nov 04 01:56:48.773778 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:56:48.773758    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf\": container with ID starting with 6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf not found: ID does not exist" containerID="6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf"
Nov 04 01:56:48.773826 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.773786    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf"} err="failed to get container status \"6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf\": rpc error: code = NotFound desc = could not find container \"6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf\": container with ID starting with 6df4489e1a67a604820ea8de321205384c1e4ceeac1aa02a3defa098d5afe6bf not found: ID does not exist"
Nov 04 01:56:48.928720 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.928679    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8c458698d-h589f"]
Nov 04 01:56:48.978282 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:48.978251    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8c458698d-h589f"]
Nov 04 01:56:50.309351 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:50.309317    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" path="/var/lib/kubelet/pods/9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7/volumes"
Nov 04 01:56:53.782812 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:53.782774    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:53.984965 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:53.984919    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:55.533030 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:55.532995    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:55.536521 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:55.536485    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5584fc7458-wcht4"
Nov 04 01:56:56.776263 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:56:56.776225    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 01:56:58.894545 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:58.894474    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-576c4d8f8-8zv54" podUID="4f781b09-233f-4bdd-9024-e52ec52b31b3" containerName="console" containerID="cri-o://1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954" gracePeriod=15
Nov 04 01:56:59.187085 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.187062    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:59.260042 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260014    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-serving-cert\") pod \"4f781b09-233f-4bdd-9024-e52ec52b31b3\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") "
Nov 04 01:56:59.260197 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260063    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-config\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-config\") pod \"4f781b09-233f-4bdd-9024-e52ec52b31b3\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") "
Nov 04 01:56:59.260197 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260128    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f781b09-233f-4bdd-9024-e52ec52b31b3-tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f781b09-233f-4bdd-9024-e52ec52b31b3-tmp\") pod \"4f781b09-233f-4bdd-9024-e52ec52b31b3\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") "
Nov 04 01:56:59.260197 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260160    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-oauth-serving-cert\") pod \"4f781b09-233f-4bdd-9024-e52ec52b31b3\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") "
Nov 04 01:56:59.260197 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260180    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-service-ca\") pod \"4f781b09-233f-4bdd-9024-e52ec52b31b3\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") "
Nov 04 01:56:59.260382 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260216    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-oauth-config\") pod \"4f781b09-233f-4bdd-9024-e52ec52b31b3\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") "
Nov 04 01:56:59.260382 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260246    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f781b09-233f-4bdd-9024-e52ec52b31b3-kube-api-access-56lr6\" (UniqueName: \"kubernetes.io/projected/4f781b09-233f-4bdd-9024-e52ec52b31b3-kube-api-access-56lr6\") pod \"4f781b09-233f-4bdd-9024-e52ec52b31b3\" (UID: \"4f781b09-233f-4bdd-9024-e52ec52b31b3\") "
Nov 04 01:56:59.260474 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260443    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f781b09-233f-4bdd-9024-e52ec52b31b3-tmp" pod "4f781b09-233f-4bdd-9024-e52ec52b31b3" (UID: "4f781b09-233f-4bdd-9024-e52ec52b31b3"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:59.260602 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260579    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-config" pod "4f781b09-233f-4bdd-9024-e52ec52b31b3" (UID: "4f781b09-233f-4bdd-9024-e52ec52b31b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:59.260602 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260590    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-oauth-serving-cert" pod "4f781b09-233f-4bdd-9024-e52ec52b31b3" (UID: "4f781b09-233f-4bdd-9024-e52ec52b31b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:59.260706 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.260647    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-service-ca" pod "4f781b09-233f-4bdd-9024-e52ec52b31b3" (UID: "4f781b09-233f-4bdd-9024-e52ec52b31b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:59.262202 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.262177    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-serving-cert" pod "4f781b09-233f-4bdd-9024-e52ec52b31b3" (UID: "4f781b09-233f-4bdd-9024-e52ec52b31b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:59.262385 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.262365    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-oauth-config" pod "4f781b09-233f-4bdd-9024-e52ec52b31b3" (UID: "4f781b09-233f-4bdd-9024-e52ec52b31b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:59.262534 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.262496    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f781b09-233f-4bdd-9024-e52ec52b31b3-kube-api-access-56lr6" pod "4f781b09-233f-4bdd-9024-e52ec52b31b3" (UID: "4f781b09-233f-4bdd-9024-e52ec52b31b3"). InnerVolumeSpecName "kube-api-access-56lr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:59.361774 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.361736    4512 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-oauth-serving-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:59.361774 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.361766    4512 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-service-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:59.361774 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.361776    4512 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-oauth-config\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:59.361996 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.361786    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56lr6\" (UniqueName: \"kubernetes.io/projected/4f781b09-233f-4bdd-9024-e52ec52b31b3-kube-api-access-56lr6\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:59.361996 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.361796    4512 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-serving-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:59.361996 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.361804    4512 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f781b09-233f-4bdd-9024-e52ec52b31b3-console-config\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:59.361996 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.361812    4512 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f781b09-233f-4bdd-9024-e52ec52b31b3-tmp\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:59.814102 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.814067    4512 generic.go:356] "Generic (PLEG): container finished" podID="4f781b09-233f-4bdd-9024-e52ec52b31b3" containerID="1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954" exitCode=0
Nov 04 01:56:59.814270 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.814111    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c4d8f8-8zv54"
Nov 04 01:56:59.814270 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.814161    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c4d8f8-8zv54" event={"ID":"4f781b09-233f-4bdd-9024-e52ec52b31b3","Type":"ContainerDied","Data":"1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954"}
Nov 04 01:56:59.814270 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.814206    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c4d8f8-8zv54" event={"ID":"4f781b09-233f-4bdd-9024-e52ec52b31b3","Type":"ContainerDied","Data":"35498a89a70307a7f66ec076f766a30c34c89af0cd774c61ecdb5bced5e810e6"}
Nov 04 01:56:59.814270 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.814220    4512 scope.go:122] "RemoveContainer" containerID="1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954"
Nov 04 01:56:59.815765 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.815737    4512 generic.go:356] "Generic (PLEG): container finished" podID="e7d3eb2e-9765-455d-b901-6596bdb56f7e" containerID="7ea8b6a28f05012a07dab156f123273663730cd0c2119c08ab6c2aa30d5bae6a" exitCode=0
Nov 04 01:56:59.815873 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.815808    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lzc9g-h2w8g" event={"ID":"e7d3eb2e-9765-455d-b901-6596bdb56f7e","Type":"ContainerDied","Data":"7ea8b6a28f05012a07dab156f123273663730cd0c2119c08ab6c2aa30d5bae6a"}
Nov 04 01:56:59.822284 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.822270    4512 scope.go:122] "RemoveContainer" containerID="1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954"
Nov 04 01:56:59.822539 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:56:59.822521    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954\": container with ID starting with 1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954 not found: ID does not exist" containerID="1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954"
Nov 04 01:56:59.822599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:56:59.822547    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954"} err="failed to get container status \"1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954\": rpc error: code = NotFound desc = could not find container \"1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954\": container with ID starting with 1b80932b42f8c85d5fe710ba32025821a4a9130c7faf9e6e118b7b2a32ba9954 not found: ID does not exist"
Nov 04 01:57:00.091948 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:00.091859    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-576c4d8f8-8zv54"]
Nov 04 01:57:00.096059 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:00.096033    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-576c4d8f8-8zv54"]
Nov 04 01:57:00.282383 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:00.282353    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-lzc9g-h2w8g"]
Nov 04 01:57:00.310601 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:00.310568    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f781b09-233f-4bdd-9024-e52ec52b31b3" path="/var/lib/kubelet/pods/4f781b09-233f-4bdd-9024-e52ec52b31b3/volumes"
Nov 04 01:57:00.361077 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:00.360995    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:00.980280 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:00.980249    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:57:01.075429 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.075349    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-service-ca-bundle\") pod \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") "
Nov 04 01:57:01.075429 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.075402    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7d3eb2e-9765-455d-b901-6596bdb56f7e-kube-api-access-r9k72\" (UniqueName: \"kubernetes.io/projected/e7d3eb2e-9765-455d-b901-6596bdb56f7e-kube-api-access-r9k72\") pod \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") "
Nov 04 01:57:01.075652 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.075440    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-trusted-ca-bundle\") pod \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") "
Nov 04 01:57:01.075652 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.075475    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7d3eb2e-9765-455d-b901-6596bdb56f7e-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e7d3eb2e-9765-455d-b901-6596bdb56f7e-archives-path\") pod \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\" (UID: \"e7d3eb2e-9765-455d-b901-6596bdb56f7e\") "
Nov 04 01:57:01.075772 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.075750    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-service-ca-bundle" pod "e7d3eb2e-9765-455d-b901-6596bdb56f7e" (UID: "e7d3eb2e-9765-455d-b901-6596bdb56f7e"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:01.076006 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.075982    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7d3eb2e-9765-455d-b901-6596bdb56f7e-archives-path" pod "e7d3eb2e-9765-455d-b901-6596bdb56f7e" (UID: "e7d3eb2e-9765-455d-b901-6596bdb56f7e"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:01.080676 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.076205    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-trusted-ca-bundle" pod "e7d3eb2e-9765-455d-b901-6596bdb56f7e" (UID: "e7d3eb2e-9765-455d-b901-6596bdb56f7e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:01.080676 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.078725    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7d3eb2e-9765-455d-b901-6596bdb56f7e-kube-api-access-r9k72" pod "e7d3eb2e-9765-455d-b901-6596bdb56f7e" (UID: "e7d3eb2e-9765-455d-b901-6596bdb56f7e"). InnerVolumeSpecName "kube-api-access-r9k72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:01.176922 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.176877    4512 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-trusted-ca-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:01.176922 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.176916    4512 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e7d3eb2e-9765-455d-b901-6596bdb56f7e-archives-path\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:01.176922 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.176929    4512 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7d3eb2e-9765-455d-b901-6596bdb56f7e-service-ca-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:01.177367 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.176942    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9k72\" (UniqueName: \"kubernetes.io/projected/e7d3eb2e-9765-455d-b901-6596bdb56f7e-kube-api-access-r9k72\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:01.826464 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.826433    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lzc9g-h2w8g"
Nov 04 01:57:01.826714 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.826430    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lzc9g-h2w8g" event={"ID":"e7d3eb2e-9765-455d-b901-6596bdb56f7e","Type":"ContainerDied","Data":"e3fcec2a2abff87025a81d319b1a140f47f7b17c6ad193209468b49363536050"}
Nov 04 01:57:01.826714 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:01.826571    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3fcec2a2abff87025a81d319b1a140f47f7b17c6ad193209468b49363536050"
Nov 04 01:57:02.051324 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:02.051289    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-lzc9g-h2w8g"]
Nov 04 01:57:02.055532 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:02.055492    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-lzc9g-h2w8g"]
Nov 04 01:57:02.309366 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:02.309337    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7d3eb2e-9765-455d-b901-6596bdb56f7e" path="/var/lib/kubelet/pods/e7d3eb2e-9765-455d-b901-6596bdb56f7e/volumes"
Nov 04 01:57:24.421022 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:24.420983    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:26.776977 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:57:26.776937    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 01:57:29.396733 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:29.396693    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:29.661481 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:29.661396    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:31.472702 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.472666    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d5979f74c-544vf"]
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.472958    4512 cpu_manager.go:414] "containerMap: removing container" podUID="e7d3eb2e-9765-455d-b901-6596bdb56f7e" containerName="insights-gathering"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.472969    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7d3eb2e-9765-455d-b901-6596bdb56f7e" containerName="insights-gathering"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.472984    4512 cpu_manager.go:414] "containerMap: removing container" podUID="4f781b09-233f-4bdd-9024-e52ec52b31b3" containerName="console"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.472989    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f781b09-233f-4bdd-9024-e52ec52b31b3" containerName="console"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.472997    4512 cpu_manager.go:414] "containerMap: removing container" podUID="6fdfc9f7-60e3-43f3-b85d-9adf12316697" containerName="console"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.473002    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fdfc9f7-60e3-43f3-b85d-9adf12316697" containerName="console"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.473017    4512 cpu_manager.go:414] "containerMap: removing container" podUID="9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" containerName="console"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.473021    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" containerName="console"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.473091    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7d3eb2e-9765-455d-b901-6596bdb56f7e" containerName="insights-gathering"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.473100    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f781b09-233f-4bdd-9024-e52ec52b31b3" containerName="console"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.473105    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fdfc9f7-60e3-43f3-b85d-9adf12316697" containerName="console"
Nov 04 01:57:31.473184 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.473113    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fffa60b-7ee8-4cdc-a8b8-c09ead0f82a7" containerName="console"
Nov 04 01:57:31.475185 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.475169    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.535195 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.535157    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d5979f74c-544vf"]
Nov 04 01:57:31.632778 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.632739    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-console-config\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.632778 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.632778    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg7bx\" (UniqueName: \"kubernetes.io/projected/bd9eec4c-bde8-44c7-817d-452deb2064e3-kube-api-access-gg7bx\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.633045 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.632796    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd9eec4c-bde8-44c7-817d-452deb2064e3-tmp\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.633045 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.632861    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-oauth-serving-cert\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.633045 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.632963    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd9eec4c-bde8-44c7-817d-452deb2064e3-console-oauth-config\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.633045 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.633023    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-service-ca\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.633238 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.633115    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-trusted-ca-bundle\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.633238 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.633150    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9eec4c-bde8-44c7-817d-452deb2064e3-console-serving-cert\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.733736 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.733702    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-service-ca\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.733933 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.733755    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-trusted-ca-bundle\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.733933 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.733786    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9eec4c-bde8-44c7-817d-452deb2064e3-console-serving-cert\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.733933 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.733833    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-console-config\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.733933 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.733861    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg7bx\" (UniqueName: \"kubernetes.io/projected/bd9eec4c-bde8-44c7-817d-452deb2064e3-kube-api-access-gg7bx\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.733933 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.733889    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd9eec4c-bde8-44c7-817d-452deb2064e3-tmp\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.733933 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.733915    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-oauth-serving-cert\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.734241 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.733943    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd9eec4c-bde8-44c7-817d-452deb2064e3-console-oauth-config\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.734437 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.734411    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd9eec4c-bde8-44c7-817d-452deb2064e3-tmp\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.734608 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.734586    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-service-ca\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.734685 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.734666    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-console-config\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.734845 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.734821    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-trusted-ca-bundle\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.734969 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.734947    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd9eec4c-bde8-44c7-817d-452deb2064e3-oauth-serving-cert\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.736403 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.736378    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9eec4c-bde8-44c7-817d-452deb2064e3-console-serving-cert\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.736531 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.736491    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd9eec4c-bde8-44c7-817d-452deb2064e3-console-oauth-config\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.794598 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.794570    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg7bx\" (UniqueName: \"kubernetes.io/projected/bd9eec4c-bde8-44c7-817d-452deb2064e3-kube-api-access-gg7bx\") pod \"console-7d5979f74c-544vf\" (UID: \"bd9eec4c-bde8-44c7-817d-452deb2064e3\") " pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:31.807429 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:31.803661    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:32.051997 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:32.051908    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:32.085213 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:32.085187    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:32.252873 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:32.252837    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:32.503620 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:32.503521    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d5979f74c-544vf"]
Nov 04 01:57:32.506310 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:57:32.506275    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd9eec4c_bde8_44c7_817d_452deb2064e3.slice/crio-ccdd1b72e0fc1deb20240720fcea4538514c44bf91daa7a3cc2c673ef036597d WatchSource:0}: Error finding container ccdd1b72e0fc1deb20240720fcea4538514c44bf91daa7a3cc2c673ef036597d: Status 404 returned error can't find the container with id ccdd1b72e0fc1deb20240720fcea4538514c44bf91daa7a3cc2c673ef036597d
Nov 04 01:57:32.950023 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:32.949940    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5979f74c-544vf" event={"ID":"bd9eec4c-bde8-44c7-817d-452deb2064e3","Type":"ContainerStarted","Data":"77e0646d0c3441c9084fa10933332fc3a002a9f4d957018ec16b507b75eb1913"}
Nov 04 01:57:32.950023 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:32.949991    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5979f74c-544vf" event={"ID":"bd9eec4c-bde8-44c7-817d-452deb2064e3","Type":"ContainerStarted","Data":"ccdd1b72e0fc1deb20240720fcea4538514c44bf91daa7a3cc2c673ef036597d"}
Nov 04 01:57:33.117195 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:33.117145    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7d5979f74c-544vf" podStartSLOduration=2.117129436 podStartE2EDuration="2.117129436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:57:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:57:33.06743255 +0000 UTC m=+114.872889183" watchObservedRunningTime="2026-11-04 01:57:33.117129436 +0000 UTC m=+114.922586066"
Nov 04 01:57:37.100422 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.100381    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 01:57:37.101061 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.100672    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="alertmanager" containerID="cri-o://844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09" gracePeriod=120
Nov 04 01:57:37.101061 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.100739    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy-metric" containerID="cri-o://efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730" gracePeriod=120
Nov 04 01:57:37.101061 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.100776    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="config-reloader" containerID="cri-o://1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f" gracePeriod=120
Nov 04 01:57:37.101061 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.100850    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="prom-label-proxy" containerID="cri-o://c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8" gracePeriod=120
Nov 04 01:57:37.101061 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.100748    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy-web" containerID="cri-o://d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a" gracePeriod=120
Nov 04 01:57:37.101061 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.100797    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy" containerID="cri-o://ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482" gracePeriod=120
Nov 04 01:57:37.974132 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.974100    4512 generic.go:356] "Generic (PLEG): container finished" podID="3da135c1-ccf8-4102-b3fb-59768928c672" containerID="c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8" exitCode=0
Nov 04 01:57:37.974132 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.974125    4512 generic.go:356] "Generic (PLEG): container finished" podID="3da135c1-ccf8-4102-b3fb-59768928c672" containerID="efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730" exitCode=0
Nov 04 01:57:37.974132 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.974131    4512 generic.go:356] "Generic (PLEG): container finished" podID="3da135c1-ccf8-4102-b3fb-59768928c672" containerID="ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482" exitCode=0
Nov 04 01:57:37.974132 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.974137    4512 generic.go:356] "Generic (PLEG): container finished" podID="3da135c1-ccf8-4102-b3fb-59768928c672" containerID="1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f" exitCode=0
Nov 04 01:57:37.974132 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.974144    4512 generic.go:356] "Generic (PLEG): container finished" podID="3da135c1-ccf8-4102-b3fb-59768928c672" containerID="844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09" exitCode=0
Nov 04 01:57:37.974425 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.974177    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerDied","Data":"c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8"}
Nov 04 01:57:37.974425 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.974246    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerDied","Data":"efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730"}
Nov 04 01:57:37.974425 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.974271    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerDied","Data":"ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482"}
Nov 04 01:57:37.974425 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.974296    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerDied","Data":"1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f"}
Nov 04 01:57:37.974425 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:37.974327    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerDied","Data":"844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09"}
Nov 04 01:57:38.356683 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.356656    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:38.487822 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.487783    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-cluster-tls-config\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488000 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.487833    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-kube-api-access-6mv6c\" (UniqueName: \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-kube-api-access-6mv6c\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488000 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.487877    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-metrics-client-ca\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488000 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.487897    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-web-config\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-web-config\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488000 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.487924    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-main-tls\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488000 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.487948    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-tls-assets\" (UniqueName: \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-tls-assets\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488000 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.487975    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488311 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.488006    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-config-out\" (UniqueName: \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-config-out\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488311 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.488051    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-main-db\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488311 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.488095    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-trusted-ca-bundle\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488311 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.488133    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488311 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.488167    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-config-volume\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-config-volume\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488311 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.488214    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-web\") pod \"3da135c1-ccf8-4102-b3fb-59768928c672\" (UID: \"3da135c1-ccf8-4102-b3fb-59768928c672\") "
Nov 04 01:57:38.488794 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.488608    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-metrics-client-ca" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:38.489108 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.489077    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-main-db" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:38.489447 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.489318    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-trusted-ca-bundle" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:38.491941 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.491882    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-web" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:57:38.491941 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.491886    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-kube-api-access-6mv6c" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "kube-api-access-6mv6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:38.492111 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.491945    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-main-tls" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:57:38.492451 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.492423    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-config-out" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:38.492622 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.492602    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-config-volume" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:57:38.492686 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.492618    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-tls-assets" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:38.492833 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.492806    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-metric" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:57:38.492914 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.492842    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:57:38.495921 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.495893    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-cluster-tls-config" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:57:38.503382 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.503350    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-web-config" pod "3da135c1-ccf8-4102-b3fb-59768928c672" (UID: "3da135c1-ccf8-4102-b3fb-59768928c672"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:57:38.589903 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589819    4512 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-cluster-tls-config\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.589903 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589847    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mv6c\" (UniqueName: \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-kube-api-access-6mv6c\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.589903 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589858    4512 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-metrics-client-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.589903 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589867    4512 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-web-config\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.589903 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589876    4512 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-main-tls\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.589903 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589886    4512 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3da135c1-ccf8-4102-b3fb-59768928c672-tls-assets\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.589903 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589896    4512 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.589903 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589904    4512 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-config-out\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.590247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589913    4512 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-main-db\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.590247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589921    4512 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da135c1-ccf8-4102-b3fb-59768928c672-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.590247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589931    4512 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.590247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589939    4512 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.590247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.589948    4512 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3da135c1-ccf8-4102-b3fb-59768928c672-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:38.981069 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.981042    4512 generic.go:356] "Generic (PLEG): container finished" podID="3da135c1-ccf8-4102-b3fb-59768928c672" containerID="d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a" exitCode=0
Nov 04 01:57:38.981224 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.981115    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerDied","Data":"d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a"}
Nov 04 01:57:38.981224 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.981126    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:38.981224 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.981168    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3da135c1-ccf8-4102-b3fb-59768928c672","Type":"ContainerDied","Data":"503a7bd7f691b03600f8f81846db9d970373f2f0d579ba4349da2114fd4987e8"}
Nov 04 01:57:38.981224 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.981201    4512 scope.go:122] "RemoveContainer" containerID="c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8"
Nov 04 01:57:38.989263 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.989242    4512 scope.go:122] "RemoveContainer" containerID="efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730"
Nov 04 01:57:38.996207 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:38.996182    4512 scope.go:122] "RemoveContainer" containerID="ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482"
Nov 04 01:57:39.002937 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.002917    4512 scope.go:122] "RemoveContainer" containerID="d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a"
Nov 04 01:57:39.009815 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.009797    4512 scope.go:122] "RemoveContainer" containerID="1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f"
Nov 04 01:57:39.016839 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.016820    4512 scope.go:122] "RemoveContainer" containerID="844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09"
Nov 04 01:57:39.023993 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.023972    4512 scope.go:122] "RemoveContainer" containerID="650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b"
Nov 04 01:57:39.031044 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.031022    4512 scope.go:122] "RemoveContainer" containerID="c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8"
Nov 04 01:57:39.031336 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:57:39.031314    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8\": container with ID starting with c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8 not found: ID does not exist" containerID="c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8"
Nov 04 01:57:39.031423 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.031351    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8"} err="failed to get container status \"c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8\": rpc error: code = NotFound desc = could not find container \"c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8\": container with ID starting with c6069ce72687c085e1a85780210a02477f2d2d6ec9567bf8683d7ad0591f92f8 not found: ID does not exist"
Nov 04 01:57:39.031423 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.031379    4512 scope.go:122] "RemoveContainer" containerID="efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730"
Nov 04 01:57:39.031656 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:57:39.031636    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730\": container with ID starting with efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730 not found: ID does not exist" containerID="efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730"
Nov 04 01:57:39.031703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.031664    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730"} err="failed to get container status \"efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730\": rpc error: code = NotFound desc = could not find container \"efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730\": container with ID starting with efe4cdb38bd54d230fc2fe7fdf0f25b692141a42b19a4cce4c0ba17394415730 not found: ID does not exist"
Nov 04 01:57:39.031703 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.031679    4512 scope.go:122] "RemoveContainer" containerID="ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482"
Nov 04 01:57:39.031912 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:57:39.031894    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482\": container with ID starting with ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482 not found: ID does not exist" containerID="ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482"
Nov 04 01:57:39.031984 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.031923    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482"} err="failed to get container status \"ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482\": rpc error: code = NotFound desc = could not find container \"ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482\": container with ID starting with ad0495f735d5e9082f605a55e515d1600bfd40a2ed90d53f8b9383585629c482 not found: ID does not exist"
Nov 04 01:57:39.031984 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.031946    4512 scope.go:122] "RemoveContainer" containerID="d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a"
Nov 04 01:57:39.032180 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:57:39.032154    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a\": container with ID starting with d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a not found: ID does not exist" containerID="d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a"
Nov 04 01:57:39.032222 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.032176    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a"} err="failed to get container status \"d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a\": rpc error: code = NotFound desc = could not find container \"d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a\": container with ID starting with d8114d1de30f5fde11c6b5e361f4377a26f49945c807f44de885b99b8de8666a not found: ID does not exist"
Nov 04 01:57:39.032222 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.032188    4512 scope.go:122] "RemoveContainer" containerID="1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f"
Nov 04 01:57:39.032408 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:57:39.032381    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f\": container with ID starting with 1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f not found: ID does not exist" containerID="1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f"
Nov 04 01:57:39.032485 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.032413    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f"} err="failed to get container status \"1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f\": rpc error: code = NotFound desc = could not find container \"1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f\": container with ID starting with 1f3949d98ea18418964fdb2012e8ca50a987f25314c6e803b7d74af82fa61b1f not found: ID does not exist"
Nov 04 01:57:39.032485 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.032426    4512 scope.go:122] "RemoveContainer" containerID="844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09"
Nov 04 01:57:39.032679 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:57:39.032663    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09\": container with ID starting with 844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09 not found: ID does not exist" containerID="844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09"
Nov 04 01:57:39.032715 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.032683    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09"} err="failed to get container status \"844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09\": rpc error: code = NotFound desc = could not find container \"844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09\": container with ID starting with 844f48830cca8d3bb3502fb2d62fcd400bc8bc3f18ca8e72ed39b22b135dae09 not found: ID does not exist"
Nov 04 01:57:39.032715 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.032697    4512 scope.go:122] "RemoveContainer" containerID="650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b"
Nov 04 01:57:39.032935 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:57:39.032921    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b\": container with ID starting with 650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b not found: ID does not exist" containerID="650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b"
Nov 04 01:57:39.032968 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.032937    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b"} err="failed to get container status \"650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b\": rpc error: code = NotFound desc = could not find container \"650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b\": container with ID starting with 650a7d4a64021bb43066432e846c07db3a8a81ce7d1b3f7cc82be6135f979c3b not found: ID does not exist"
Nov 04 01:57:39.147755 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.147717    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 01:57:39.197177 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.197080    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 01:57:39.198128 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198102    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 01:57:39.198500 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198481    4512 cpu_manager.go:414] "containerMap: removing container" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy-metric"
Nov 04 01:57:39.198612 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198520    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy-metric"
Nov 04 01:57:39.198612 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198535    4512 cpu_manager.go:414] "containerMap: removing container" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="prom-label-proxy"
Nov 04 01:57:39.198612 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198542    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="prom-label-proxy"
Nov 04 01:57:39.198612 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198556    4512 cpu_manager.go:414] "containerMap: removing container" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="alertmanager"
Nov 04 01:57:39.198612 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198563    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="alertmanager"
Nov 04 01:57:39.198612 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198577    4512 cpu_manager.go:414] "containerMap: removing container" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="config-reloader"
Nov 04 01:57:39.198612 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198584    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="config-reloader"
Nov 04 01:57:39.198612 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198600    4512 cpu_manager.go:414] "containerMap: removing container" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy-web"
Nov 04 01:57:39.198612 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198607    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy-web"
Nov 04 01:57:39.199007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198630    4512 cpu_manager.go:414] "containerMap: removing container" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy"
Nov 04 01:57:39.199007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198637    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy"
Nov 04 01:57:39.199007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198646    4512 cpu_manager.go:414] "containerMap: removing container" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="init-config-reloader"
Nov 04 01:57:39.199007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198652    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="init-config-reloader"
Nov 04 01:57:39.199007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198734    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy-metric"
Nov 04 01:57:39.199007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198746    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="prom-label-proxy"
Nov 04 01:57:39.199007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198754    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="alertmanager"
Nov 04 01:57:39.199007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198764    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="config-reloader"
Nov 04 01:57:39.199007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198773    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy-web"
Nov 04 01:57:39.199007 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.198783    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" containerName="kube-rbac-proxy"
Nov 04 01:57:39.202463 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.202442    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.255247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.255046    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 01:57:39.255247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.255117    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 01:57:39.255247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.255224    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 01:57:39.255247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.255236    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 01:57:39.255247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.255248    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 01:57:39.255247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.255236    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t678k\""
Nov 04 01:57:39.255247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.255232    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 01:57:39.255247 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.255282    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 01:57:39.256031 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.255425    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 01:57:39.295317 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295281    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295317 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295321    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295481 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295344    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d12e508-96a3-4551-84e5-da9736a50368-config-out\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295481 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295427    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d12e508-96a3-4551-84e5-da9736a50368-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295481 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295454    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d12e508-96a3-4551-84e5-da9736a50368-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295605 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295549    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295605 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295583    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d12e508-96a3-4551-84e5-da9736a50368-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295663 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295613    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-web-config\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295691 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295668    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5qzk\" (UniqueName: \"kubernetes.io/projected/5d12e508-96a3-4551-84e5-da9736a50368-kube-api-access-h5qzk\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295722 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295691    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295754 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295728    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d12e508-96a3-4551-84e5-da9736a50368-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295754 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295746    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.295811 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.295767    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.304596 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.304573    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 01:57:39.317272 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.317246    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 01:57:39.396213 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396174    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d12e508-96a3-4551-84e5-da9736a50368-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396222    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396240    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d12e508-96a3-4551-84e5-da9736a50368-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396257    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-web-config\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396295    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qzk\" (UniqueName: \"kubernetes.io/projected/5d12e508-96a3-4551-84e5-da9736a50368-kube-api-access-h5qzk\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396315    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396329    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d12e508-96a3-4551-84e5-da9736a50368-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396350    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396374    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396394    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396415    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396453    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d12e508-96a3-4551-84e5-da9736a50368-config-out\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396478    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d12e508-96a3-4551-84e5-da9736a50368-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.396762 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.396726    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d12e508-96a3-4551-84e5-da9736a50368-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.397975 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.397820    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d12e508-96a3-4551-84e5-da9736a50368-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.397975 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.397930    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d12e508-96a3-4551-84e5-da9736a50368-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.399748 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.399724    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.400313 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.400202    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.400313 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.400238    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-web-config\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.400313 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.400283    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.400313 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.400294    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.400497 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.400366    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d12e508-96a3-4551-84e5-da9736a50368-config-out\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.400497 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.400378    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d12e508-96a3-4551-84e5-da9736a50368-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.401097 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.401071    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.401373 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.401354    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d12e508-96a3-4551-84e5-da9736a50368-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.458149 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.458117    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5qzk\" (UniqueName: \"kubernetes.io/projected/5d12e508-96a3-4551-84e5-da9736a50368-kube-api-access-h5qzk\") pod \"alertmanager-main-0\" (UID: \"5d12e508-96a3-4551-84e5-da9736a50368\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.518306 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.518212    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 01:57:39.945744 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.945717    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 01:57:39.948257 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:57:39.948228    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d12e508_96a3_4551_84e5_da9736a50368.slice/crio-a93633d5bdff11a7c59b8e1ae924f45797a635c883da02a8d4dc1ee350c5d930 WatchSource:0}: Error finding container a93633d5bdff11a7c59b8e1ae924f45797a635c883da02a8d4dc1ee350c5d930: Status 404 returned error can't find the container with id a93633d5bdff11a7c59b8e1ae924f45797a635c883da02a8d4dc1ee350c5d930
Nov 04 01:57:39.986035 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:39.986004    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d12e508-96a3-4551-84e5-da9736a50368","Type":"ContainerStarted","Data":"a93633d5bdff11a7c59b8e1ae924f45797a635c883da02a8d4dc1ee350c5d930"}
Nov 04 01:57:40.312209 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:40.312171    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3da135c1-ccf8-4102-b3fb-59768928c672" path="/var/lib/kubelet/pods/3da135c1-ccf8-4102-b3fb-59768928c672/volumes"
Nov 04 01:57:40.992679 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:40.992641    4512 generic.go:356] "Generic (PLEG): container finished" podID="5d12e508-96a3-4551-84e5-da9736a50368" containerID="6af0cf276f7eec1a6af9986f8dba484f104891b79e77ad35c0ac9ec0ff111e18" exitCode=0
Nov 04 01:57:40.993078 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:40.992713    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d12e508-96a3-4551-84e5-da9736a50368","Type":"ContainerDied","Data":"6af0cf276f7eec1a6af9986f8dba484f104891b79e77ad35c0ac9ec0ff111e18"}
Nov 04 01:57:42.008213 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.008176    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d12e508-96a3-4551-84e5-da9736a50368","Type":"ContainerStarted","Data":"3fbec7c9a242e536e68da067e2272d8b7c20be752e30b3f3aeafad542c769d94"}
Nov 04 01:57:42.008598 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.008266    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d12e508-96a3-4551-84e5-da9736a50368","Type":"ContainerStarted","Data":"0c384097ae209921805e7174cc2f33fbcc60b309233ced8e9b7477489ab7c266"}
Nov 04 01:57:42.008598 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.008291    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d12e508-96a3-4551-84e5-da9736a50368","Type":"ContainerStarted","Data":"0c22ca2dd9acec97fdd00db8d7e9230c1851bdf87be0354526bc6710f5dd62b0"}
Nov 04 01:57:42.008598 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.008318    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d12e508-96a3-4551-84e5-da9736a50368","Type":"ContainerStarted","Data":"d6a977cb3fa240bc0125feab11179a33ddf992b6f33bfeb56166be66cc3810d9"}
Nov 04 01:57:42.008598 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.008342    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d12e508-96a3-4551-84e5-da9736a50368","Type":"ContainerStarted","Data":"c940024e2edb09b50e5381ca0db9aa8bfb7fac0d30fc5d2e6de87743421826e2"}
Nov 04 01:57:42.008598 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.008363    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d12e508-96a3-4551-84e5-da9736a50368","Type":"ContainerStarted","Data":"5647b6282c2b3afac390c9a53ea9c96525b1eb1addb5754e8f7c26893d58ab23"}
Nov 04 01:57:42.085312 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.085269    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:42.085490 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.085353    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:42.090022 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.089999    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:42.139791 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.139754    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:42.145187 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.145138    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.130962605 podStartE2EDuration="3.145124441s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.161836ms" isStatefulPod=false podCreationTimestamp="2026-11-04 01:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:57:42.143954364 +0000 UTC m=+123.949410996" watchObservedRunningTime="2026-11-04 01:57:42.145124441 +0000 UTC m=+123.950581071"
Nov 04 01:57:42.343460 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:42.343367    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:43.016009 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:43.015980    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d5979f74c-544vf"
Nov 04 01:57:43.183756 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:43.183724    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d574477cd-phdzq"]
Nov 04 01:57:53.806680 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:53.806591    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:54.058741 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:57:54.058651    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:56.776218 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:57:56.776179    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 01:58:08.203470 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.203422    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5d574477cd-phdzq" podUID="6ebd3552-420c-4bff-a1ec-16228a2d5c8a" containerName="console" containerID="cri-o://4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f" gracePeriod=15
Nov 04 01:58:08.443357 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.443334    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:58:08.513850 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.513823    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-serving-cert\") pod \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") "
Nov 04 01:58:08.513968 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.513861    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-kube-api-access-4v2pj\" (UniqueName: \"kubernetes.io/projected/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-kube-api-access-4v2pj\") pod \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") "
Nov 04 01:58:08.513968 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.513888    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-oauth-config\") pod \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") "
Nov 04 01:58:08.513968 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.513918    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-config\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-config\") pod \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") "
Nov 04 01:58:08.513968 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.513947    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-tmp\") pod \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") "
Nov 04 01:58:08.514138 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.513999    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-oauth-serving-cert\") pod \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") "
Nov 04 01:58:08.514138 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.514038    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-trusted-ca-bundle\") pod \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") "
Nov 04 01:58:08.514138 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.514078    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-service-ca\") pod \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\" (UID: \"6ebd3552-420c-4bff-a1ec-16228a2d5c8a\") "
Nov 04 01:58:08.514747 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.514706    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-config" pod "6ebd3552-420c-4bff-a1ec-16228a2d5c8a" (UID: "6ebd3552-420c-4bff-a1ec-16228a2d5c8a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:08.515036 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.515009    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-tmp" pod "6ebd3552-420c-4bff-a1ec-16228a2d5c8a" (UID: "6ebd3552-420c-4bff-a1ec-16228a2d5c8a"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:08.515336 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.515190    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-oauth-serving-cert" pod "6ebd3552-420c-4bff-a1ec-16228a2d5c8a" (UID: "6ebd3552-420c-4bff-a1ec-16228a2d5c8a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:08.515431 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.515412    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-service-ca" pod "6ebd3552-420c-4bff-a1ec-16228a2d5c8a" (UID: "6ebd3552-420c-4bff-a1ec-16228a2d5c8a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:08.518542 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.518338    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-oauth-config" pod "6ebd3552-420c-4bff-a1ec-16228a2d5c8a" (UID: "6ebd3552-420c-4bff-a1ec-16228a2d5c8a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:08.520690 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.520634    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-trusted-ca-bundle" pod "6ebd3552-420c-4bff-a1ec-16228a2d5c8a" (UID: "6ebd3552-420c-4bff-a1ec-16228a2d5c8a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:08.520877 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.520858    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-serving-cert" pod "6ebd3552-420c-4bff-a1ec-16228a2d5c8a" (UID: "6ebd3552-420c-4bff-a1ec-16228a2d5c8a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:08.521078 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.521053    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-kube-api-access-4v2pj" pod "6ebd3552-420c-4bff-a1ec-16228a2d5c8a" (UID: "6ebd3552-420c-4bff-a1ec-16228a2d5c8a"). InnerVolumeSpecName "kube-api-access-4v2pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:08.615981 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.615940    4512 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-config\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:08.615981 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.615973    4512 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-tmp\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:08.615981 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.615988    4512 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-oauth-serving-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:08.616213 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.616002    4512 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-trusted-ca-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:08.616213 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.616015    4512 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-service-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:08.616213 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.616026    4512 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-serving-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:08.616213 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.616037    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v2pj\" (UniqueName: \"kubernetes.io/projected/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-kube-api-access-4v2pj\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:08.616213 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:08.616050    4512 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ebd3552-420c-4bff-a1ec-16228a2d5c8a-console-oauth-config\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:09.107624 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:09.107590    4512 generic.go:356] "Generic (PLEG): container finished" podID="6ebd3552-420c-4bff-a1ec-16228a2d5c8a" containerID="4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f" exitCode=0
Nov 04 01:58:09.107802 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:09.107635    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d574477cd-phdzq" event={"ID":"6ebd3552-420c-4bff-a1ec-16228a2d5c8a","Type":"ContainerDied","Data":"4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f"}
Nov 04 01:58:09.107802 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:09.107645    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d574477cd-phdzq"
Nov 04 01:58:09.107802 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:09.107697    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d574477cd-phdzq" event={"ID":"6ebd3552-420c-4bff-a1ec-16228a2d5c8a","Type":"ContainerDied","Data":"6ec85d6b2ff7829ca4d7729aba7014fb5ee6b05bc2063051bbfc85381b5692dd"}
Nov 04 01:58:09.107802 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:09.107716    4512 scope.go:122] "RemoveContainer" containerID="4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f"
Nov 04 01:58:09.116122 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:09.116100    4512 scope.go:122] "RemoveContainer" containerID="4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f"
Nov 04 01:58:09.116401 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:58:09.116381    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f\": container with ID starting with 4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f not found: ID does not exist" containerID="4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f"
Nov 04 01:58:09.116453 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:09.116410    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f"} err="failed to get container status \"4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f\": rpc error: code = NotFound desc = could not find container \"4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f\": container with ID starting with 4ec2d0f0fce6ff30b25c3d89cd5cc8df17a286964633a88313500784bf9fc93f not found: ID does not exist"
Nov 04 01:58:09.273129 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:09.273093    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d574477cd-phdzq"]
Nov 04 01:58:09.322238 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:09.322205    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d574477cd-phdzq"]
Nov 04 01:58:10.310173 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:10.310140    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ebd3552-420c-4bff-a1ec-16228a2d5c8a" path="/var/lib/kubelet/pods/6ebd3552-420c-4bff-a1ec-16228a2d5c8a/volumes"
Nov 04 01:58:24.766599 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:24.766561    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:26.777382 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:58:26.777344    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 01:58:53.807039 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:58:53.807002    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:56.776095 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:58:56.776059    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 01:59:23.715382 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:23.715281    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:25.125830 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:25.125792    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:25.475461 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:25.475354    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:26.775947 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:59:26.775913    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 01:59:30.277814 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:30.277777    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:31.238181 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:31.238138    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:56.342286 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:56.342251    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:56.777157 ip-10-0-142-191 kubenswrapper[4512]: E1104 01:59:56.777114    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 01:59:58.728025 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.727987    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:58.840429 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.840396    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"]
Nov 04 01:59:58.840733 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.840719    4512 cpu_manager.go:414] "containerMap: removing container" podUID="6ebd3552-420c-4bff-a1ec-16228a2d5c8a" containerName="console"
Nov 04 01:59:58.840788 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.840735    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ebd3552-420c-4bff-a1ec-16228a2d5c8a" containerName="console"
Nov 04 01:59:58.840843 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.840802    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ebd3552-420c-4bff-a1ec-16228a2d5c8a" containerName="console"
Nov 04 01:59:58.843756 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.843740    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:58.919397 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.919369    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:58.919554 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.919409    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:58.919554 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.919530    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44nh5\" (UniqueName: \"kubernetes.io/projected/ef82cf23-0867-4c91-80d1-11fe242707a8-kube-api-access-44nh5\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:58.945487 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.945465    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 04 01:59:58.945487 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.945465    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-d8w2t\""
Nov 04 01:59:58.945487 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.945468    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 04 01:59:58.946617 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:58.946598    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"]
Nov 04 01:59:59.020519 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.020486    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:59.020661 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.020538    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:59.020661 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.020575    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44nh5\" (UniqueName: \"kubernetes.io/projected/ef82cf23-0867-4c91-80d1-11fe242707a8-kube-api-access-44nh5\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:59.020838 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.020819    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:59.020882 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.020852    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:59.081232 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.081196    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44nh5\" (UniqueName: \"kubernetes.io/projected/ef82cf23-0867-4c91-80d1-11fe242707a8-kube-api-access-44nh5\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:59.159018 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.158982    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 01:59:59.335165 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.335092    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"]
Nov 04 01:59:59.338663 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.338645    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.390907 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.390879    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-txwqj\""
Nov 04 01:59:59.397013 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.396987    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"]
Nov 04 01:59:59.406601 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.406574    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:59.423372 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.423349    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.423470 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.423386    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.423543 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.423520    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzj8t\" (UniqueName: \"kubernetes.io/projected/d5de9d8b-4ca3-488a-a727-54f0204173d2-kube-api-access-gzj8t\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.431740 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.431719    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"]
Nov 04 01:59:59.435034 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.435020    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.490909 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.490886    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-r6rnv\""
Nov 04 01:59:59.508287 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.508263    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"]
Nov 04 01:59:59.523957 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.523931    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzj8t\" (UniqueName: \"kubernetes.io/projected/d5de9d8b-4ca3-488a-a727-54f0204173d2-kube-api-access-gzj8t\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.524100 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.523993    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvsp4\" (UniqueName: \"kubernetes.io/projected/6452875f-0203-4b2e-8593-7e47cf9710d8-kube-api-access-qvsp4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.524100 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.524032    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.524208 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.524130    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.524208 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.524163    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.524208 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.524188    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.524458 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.524439    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.524547 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.524486    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.533453 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.533430    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"]
Nov 04 01:59:59.536868 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.536850    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.568961 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.568940    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"]
Nov 04 01:59:59.572276 ip-10-0-142-191 kubenswrapper[4512]: W1104 01:59:59.572228    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef82cf23_0867_4c91_80d1_11fe242707a8.slice/crio-5c8298a22c6a4a2c03817a19bf9d7c4581c1f9ee2ff0afc889f1094baf69650b WatchSource:0}: Error finding container 5c8298a22c6a4a2c03817a19bf9d7c4581c1f9ee2ff0afc889f1094baf69650b: Status 404 returned error can't find the container with id 5c8298a22c6a4a2c03817a19bf9d7c4581c1f9ee2ff0afc889f1094baf69650b
Nov 04 01:59:59.587298 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.587246    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzj8t\" (UniqueName: \"kubernetes.io/projected/d5de9d8b-4ca3-488a-a727-54f0204173d2-kube-api-access-gzj8t\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.590989 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.590968    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-qq6pd\""
Nov 04 01:59:59.595735 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.595715    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"]
Nov 04 01:59:59.624944 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.624918    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.625032 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.624960    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvsp4\" (UniqueName: \"kubernetes.io/projected/6452875f-0203-4b2e-8593-7e47cf9710d8-kube-api-access-qvsp4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.625075 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.625027    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.625075 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.625060    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.625157 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.625111    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.625157 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.625138    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2w5n\" (UniqueName: \"kubernetes.io/projected/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-kube-api-access-p2w5n\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.625405 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.625385    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.625467 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.625454    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.635850 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.635829    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"]
Nov 04 01:59:59.639404 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.639388    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 01:59:59.654395 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.654378    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 01:59:59.686558 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.686530    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvsp4\" (UniqueName: \"kubernetes.io/projected/6452875f-0203-4b2e-8593-7e47cf9710d8-kube-api-access-qvsp4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.692942 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.692914    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-mzqkb\""
Nov 04 01:59:59.700447 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.700419    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"]
Nov 04 01:59:59.726016 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.725990    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.726119 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.726049    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tljq\" (UniqueName: \"kubernetes.io/projected/a4405cc0-326d-4057-b87b-0b7c5efc10ae-kube-api-access-9tljq\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 01:59:59.726172 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.726160    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 01:59:59.726217 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.726178    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 01:59:59.726287 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.726261    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2w5n\" (UniqueName: \"kubernetes.io/projected/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-kube-api-access-p2w5n\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.726372 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.726357    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.726423 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.726393    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.726636 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.726620    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.751073 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.751052    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 01:59:59.787321 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.787290    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2w5n\" (UniqueName: \"kubernetes.io/projected/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-kube-api-access-p2w5n\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.827226 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.827197    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tljq\" (UniqueName: \"kubernetes.io/projected/a4405cc0-326d-4057-b87b-0b7c5efc10ae-kube-api-access-9tljq\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 01:59:59.827350 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.827239    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 01:59:59.827350 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.827261    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 01:59:59.827593 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.827571    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 01:59:59.827674 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.827645    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 01:59:59.851920 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.851862    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 01:59:59.875292 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.875271    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"]
Nov 04 01:59:59.879083 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.879066    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 01:59:59.888123 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.888096    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tljq\" (UniqueName: \"kubernetes.io/projected/a4405cc0-326d-4057-b87b-0b7c5efc10ae-kube-api-access-9tljq\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 01:59:59.927796 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.927767    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 01:59:59.927917 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.927807    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 01:59:59.927917 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.927869    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fmrl\" (UniqueName: \"kubernetes.io/projected/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-kube-api-access-7fmrl\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 01:59:59.931306 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.931284    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-89vnh\""
Nov 04 01:59:59.937385 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.937363    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"]
Nov 04 01:59:59.955707 ip-10-0-142-191 kubenswrapper[4512]: I1104 01:59:59.955680    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 02:00:00.028708 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.028681    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 02:00:00.028847 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.028722    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 02:00:00.028847 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.028751    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fmrl\" (UniqueName: \"kubernetes.io/projected/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-kube-api-access-7fmrl\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 02:00:00.029161 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.029139    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 02:00:00.029161 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.029152    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 02:00:00.029579 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.029533    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"]
Nov 04 02:00:00.032165 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:00.032135    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5de9d8b_4ca3_488a_a727_54f0204173d2.slice/crio-108d861fefd733e4e4e85c299b6685537d34a860a1e456c50f95218eac8d26cc WatchSource:0}: Error finding container 108d861fefd733e4e4e85c299b6685537d34a860a1e456c50f95218eac8d26cc: Status 404 returned error can't find the container with id 108d861fefd733e4e4e85c299b6685537d34a860a1e456c50f95218eac8d26cc
Nov 04 02:00:00.090013 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.089985    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fmrl\" (UniqueName: \"kubernetes.io/projected/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-kube-api-access-7fmrl\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 02:00:00.123228 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.123202    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"]
Nov 04 02:00:00.125664 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:00.125635    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6452875f_0203_4b2e_8593_7e47cf9710d8.slice/crio-a709f90dafe47d01c6c893b6036054314779d09db0eab2393e6b250d7b27aefd WatchSource:0}: Error finding container a709f90dafe47d01c6c893b6036054314779d09db0eab2393e6b250d7b27aefd: Status 404 returned error can't find the container with id a709f90dafe47d01c6c893b6036054314779d09db0eab2393e6b250d7b27aefd
Nov 04 02:00:00.197001 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.196976    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 02:00:00.222878 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.222850    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"]
Nov 04 02:00:00.226182 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:00.226155    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e91bb3_552f_4d57_ad3d_41c1b458e09f.slice/crio-55c3eef83c3255539b10686813bef9d74bb4d8d60e9eb5b0b1ebfe9f4d717942 WatchSource:0}: Error finding container 55c3eef83c3255539b10686813bef9d74bb4d8d60e9eb5b0b1ebfe9f4d717942: Status 404 returned error can't find the container with id 55c3eef83c3255539b10686813bef9d74bb4d8d60e9eb5b0b1ebfe9f4d717942
Nov 04 02:00:00.325668 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.325641    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"]
Nov 04 02:00:00.328283 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:00.328256    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4405cc0_326d_4057_b87b_0b7c5efc10ae.slice/crio-51d6e86584a4178cb7ffed2d98062eb0ce08046028dfa7f12d87f5fce241c1df WatchSource:0}: Error finding container 51d6e86584a4178cb7ffed2d98062eb0ce08046028dfa7f12d87f5fce241c1df: Status 404 returned error can't find the container with id 51d6e86584a4178cb7ffed2d98062eb0ce08046028dfa7f12d87f5fce241c1df
Nov 04 02:00:00.500188 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.500152    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774" event={"ID":"6452875f-0203-4b2e-8593-7e47cf9710d8","Type":"ContainerStarted","Data":"a709f90dafe47d01c6c893b6036054314779d09db0eab2393e6b250d7b27aefd"}
Nov 04 02:00:00.501660 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.501623    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj" event={"ID":"d5de9d8b-4ca3-488a-a727-54f0204173d2","Type":"ContainerStarted","Data":"108d861fefd733e4e4e85c299b6685537d34a860a1e456c50f95218eac8d26cc"}
Nov 04 02:00:00.502893 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.502867    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5" event={"ID":"ef82cf23-0867-4c91-80d1-11fe242707a8","Type":"ContainerStarted","Data":"5c8298a22c6a4a2c03817a19bf9d7c4581c1f9ee2ff0afc889f1094baf69650b"}
Nov 04 02:00:00.504162 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.504139    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c" event={"ID":"a6e91bb3-552f-4d57-ad3d-41c1b458e09f","Type":"ContainerStarted","Data":"55c3eef83c3255539b10686813bef9d74bb4d8d60e9eb5b0b1ebfe9f4d717942"}
Nov 04 02:00:00.505659 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.505628    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq" event={"ID":"a4405cc0-326d-4057-b87b-0b7c5efc10ae","Type":"ContainerStarted","Data":"51d6e86584a4178cb7ffed2d98062eb0ce08046028dfa7f12d87f5fce241c1df"}
Nov 04 02:00:00.569710 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:00.569683    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"]
Nov 04 02:00:00.572787 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:00.572761    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d2cdf6_91b5_43e9_87e1_2c4d711bcc2b.slice/crio-df0ee1a64ee5962f53233ffb0ba0b20cbcda69c63cb4c4073782c91da3b14ca3 WatchSource:0}: Error finding container df0ee1a64ee5962f53233ffb0ba0b20cbcda69c63cb4c4073782c91da3b14ca3: Status 404 returned error can't find the container with id df0ee1a64ee5962f53233ffb0ba0b20cbcda69c63cb4c4073782c91da3b14ca3
Nov 04 02:00:01.514434 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:01.514400    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb" event={"ID":"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b","Type":"ContainerStarted","Data":"df0ee1a64ee5962f53233ffb0ba0b20cbcda69c63cb4c4073782c91da3b14ca3"}
Nov 04 02:00:03.424138 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.424102    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"]
Nov 04 02:00:03.428273 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.428245    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:03.461971 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.461727    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:03.461971 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.461798    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:03.461971 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.461852    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7phc\" (UniqueName: \"kubernetes.io/projected/2461de17-c237-444b-9d68-a36b63e2889e-kube-api-access-k7phc\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:03.535519 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.533655    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-4sn7l\""
Nov 04 02:00:03.535519 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.535433    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"]
Nov 04 02:00:03.563665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.562931    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7phc\" (UniqueName: \"kubernetes.io/projected/2461de17-c237-444b-9d68-a36b63e2889e-kube-api-access-k7phc\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:03.563665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.563035    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:03.563665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.563086    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:03.563665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.563547    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:03.563979 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.563703    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:03.626119 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.626088    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7phc\" (UniqueName: \"kubernetes.io/projected/2461de17-c237-444b-9d68-a36b63e2889e-kube-api-access-k7phc\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:03.753054 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:03.753016    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:04.289319 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:04.289277    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"]
Nov 04 02:00:04.293609 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:04.293567    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2461de17_c237_444b_9d68_a36b63e2889e.slice/crio-58cd900bdc8fa5f19e57481f1881314df9db853d7597383f4a09cac949ac4072 WatchSource:0}: Error finding container 58cd900bdc8fa5f19e57481f1881314df9db853d7597383f4a09cac949ac4072: Status 404 returned error can't find the container with id 58cd900bdc8fa5f19e57481f1881314df9db853d7597383f4a09cac949ac4072
Nov 04 02:00:04.537622 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:04.537523    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4" event={"ID":"2461de17-c237-444b-9d68-a36b63e2889e","Type":"ContainerStarted","Data":"58cd900bdc8fa5f19e57481f1881314df9db853d7597383f4a09cac949ac4072"}
Nov 04 02:00:04.630235 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:04.630126    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:04.910881 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:04.910778    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:05.110550 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.109454    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"]
Nov 04 02:00:05.114767 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.113790    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.166452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.166382    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 02:00:05.166637 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.166468    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 02:00:05.166637 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.166555    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 02:00:05.179588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.179081    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.179588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.179128    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.179588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.179160    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.179588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.179222    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.179588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.179248    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.179588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.179296    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr5hv\" (UniqueName: \"kubernetes.io/projected/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-kube-api-access-jr5hv\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.179588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.179335    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.179588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.179361    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.179588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.179397    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.190035 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.189994    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wgllv\""
Nov 04 02:00:05.218000 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.217971    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"]
Nov 04 02:00:05.269659 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.269601    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:05.281486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.280008    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.281486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.280052    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.281486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.280089    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.281486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.280148    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.281486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.280174    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.281486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.280224    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr5hv\" (UniqueName: \"kubernetes.io/projected/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-kube-api-access-jr5hv\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.281486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.280265    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.281486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.280291    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.281486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.280325    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.281486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.280809    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.283264 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.283239    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.284389 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.283526    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.284389 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.284102    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.285259 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.284936    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.286105 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.285924    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.289687 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.289661    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.345533 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.345430    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.350934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.350903    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr5hv\" (UniqueName: \"kubernetes.io/projected/8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2-kube-api-access-jr5hv\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-r74jj\" (UID: \"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.428794 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.428716    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"]
Nov 04 02:00:05.432556 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.432533    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.433388 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.433240    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:05.475828 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.475790    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:05.481720 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.481690    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.481843 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.481737    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.481843 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.481769    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.481843 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.481803    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqg8z\" (UniqueName: \"kubernetes.io/projected/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-kube-api-access-jqg8z\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.482011 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.481855    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.482011 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.481902    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.482011 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.481929    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.482011 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.481954    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.482011 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.481983    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.496231 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.496196    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"]
Nov 04 02:00:05.518391 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.518216    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-78ttk\""
Nov 04 02:00:05.583077 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.583037    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.583527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.583107    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.583527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.583144    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.583527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.583168    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.583527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.583197    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.583527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.583273    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.583527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.583301    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.583527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.583331    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.583527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.583365    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqg8z\" (UniqueName: \"kubernetes.io/projected/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-kube-api-access-jqg8z\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.584080 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.584053    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.584312 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.584292    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.585220 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.585195    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.585468 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.585442    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.586201 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.585919    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.589294 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.589249    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.592715 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.592674    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.645252 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.645192    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqg8z\" (UniqueName: \"kubernetes.io/projected/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-kube-api-access-jqg8z\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.646487 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.646436    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2bbd606-cb7b-49c8-883d-aa3ae68d5d17-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-c554b664c-9n829\" (UID: \"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.680020 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.679896    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:05.750575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.750536    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:05.901490 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:05.901424    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"]
Nov 04 02:00:05.906235 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:05.906203    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec5e90c_2c2e_40ae_98eb_2ff62195a4a2.slice/crio-fae8f4b6eadee03e21da5dc01a13c4fc5716c715b7b64ff552698d98f777dab1 WatchSource:0}: Error finding container fae8f4b6eadee03e21da5dc01a13c4fc5716c715b7b64ff552698d98f777dab1: Status 404 returned error can't find the container with id fae8f4b6eadee03e21da5dc01a13c4fc5716c715b7b64ff552698d98f777dab1
Nov 04 02:00:06.046615 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.046575    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"]
Nov 04 02:00:06.049779 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.049756    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.076179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.076153    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"]
Nov 04 02:00:06.079056 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.079035    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.088691 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.088404    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.088691 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.088476    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bss8\" (UniqueName: \"kubernetes.io/projected/885425cc-20c2-4f74-b832-a5fe841098d0-kube-api-access-9bss8\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.088691 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.088562    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.108176 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.108110    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"]
Nov 04 02:00:06.112206 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.112173    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"]
Nov 04 02:00:06.136431 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.136406    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.140371 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.140325    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"]
Nov 04 02:00:06.147319 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.147003    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"]
Nov 04 02:00:06.155593 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:06.155570    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2bbd606_cb7b_49c8_883d_aa3ae68d5d17.slice/crio-c923d8183f7f12b1cdff52d25234adbde52e9ff5083d3f6713ee1653e98613b2 WatchSource:0}: Error finding container c923d8183f7f12b1cdff52d25234adbde52e9ff5083d3f6713ee1653e98613b2: Status 404 returned error can't find the container with id c923d8183f7f12b1cdff52d25234adbde52e9ff5083d3f6713ee1653e98613b2
Nov 04 02:00:06.185396 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.185323    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"]
Nov 04 02:00:06.189150 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.189130    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.189258 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.189174    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bss8\" (UniqueName: \"kubernetes.io/projected/885425cc-20c2-4f74-b832-a5fe841098d0-kube-api-access-9bss8\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.189258 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.189201    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.189258 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.189217    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.189258 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.189247    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.189476 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.189261    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.189476 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.189286    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4kq2\" (UniqueName: \"kubernetes.io/projected/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-kube-api-access-m4kq2\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.189476 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.189311    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.189476 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.189331    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kd2w\" (UniqueName: \"kubernetes.io/projected/54cc3fcf-b55e-49f6-925b-3d9b62b94009-kube-api-access-9kd2w\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.190088 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.190065    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.190528 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.190492    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.254604 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.254481    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bss8\" (UniqueName: \"kubernetes.io/projected/885425cc-20c2-4f74-b832-a5fe841098d0-kube-api-access-9bss8\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.312575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.309498    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.312575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.309576    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.312575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.309624    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.312575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.309655    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4kq2\" (UniqueName: \"kubernetes.io/projected/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-kube-api-access-m4kq2\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.312575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.309700    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kd2w\" (UniqueName: \"kubernetes.io/projected/54cc3fcf-b55e-49f6-925b-3d9b62b94009-kube-api-access-9kd2w\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.312575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.309756    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.312575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.310183    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.312575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.310575    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.312575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.310811    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.312575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.310998    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.369117 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.369075    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:06.372405 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.372373    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kd2w\" (UniqueName: \"kubernetes.io/projected/54cc3fcf-b55e-49f6-925b-3d9b62b94009-kube-api-access-9kd2w\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.386737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.386672    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4kq2\" (UniqueName: \"kubernetes.io/projected/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-kube-api-access-m4kq2\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.397000 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.396977    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:06.469096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.468606    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:06.580087 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.579946    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829" event={"ID":"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17","Type":"ContainerStarted","Data":"c923d8183f7f12b1cdff52d25234adbde52e9ff5083d3f6713ee1653e98613b2"}
Nov 04 02:00:06.583703 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.583630    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj" event={"ID":"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2","Type":"ContainerStarted","Data":"fae8f4b6eadee03e21da5dc01a13c4fc5716c715b7b64ff552698d98f777dab1"}
Nov 04 02:00:06.769187 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.769087    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"]
Nov 04 02:00:06.816081 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.815930    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"]
Nov 04 02:00:06.820093 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:06.820063    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54cc3fcf_b55e_49f6_925b_3d9b62b94009.slice/crio-62402bafec5905602535fd1cd9dd22f514496e42d8e70c770cd14f4a0bdf4cef WatchSource:0}: Error finding container 62402bafec5905602535fd1cd9dd22f514496e42d8e70c770cd14f4a0bdf4cef: Status 404 returned error can't find the container with id 62402bafec5905602535fd1cd9dd22f514496e42d8e70c770cd14f4a0bdf4cef
Nov 04 02:00:06.897462 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:06.897432    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"]
Nov 04 02:00:06.901098 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:06.901067    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a0d7d21_bc96_46eb_8370_bdb0cd57620b.slice/crio-9e0a435224230429dcd005251e3b5d4ca3e8082f3ef21254e44d2b9776fbc40e WatchSource:0}: Error finding container 9e0a435224230429dcd005251e3b5d4ca3e8082f3ef21254e44d2b9776fbc40e: Status 404 returned error can't find the container with id 9e0a435224230429dcd005251e3b5d4ca3e8082f3ef21254e44d2b9776fbc40e
Nov 04 02:00:07.592594 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:07.592545    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm" event={"ID":"8a0d7d21-bc96-46eb-8370-bdb0cd57620b","Type":"ContainerStarted","Data":"9e0a435224230429dcd005251e3b5d4ca3e8082f3ef21254e44d2b9776fbc40e"}
Nov 04 02:00:07.597065 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:07.597035    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq" event={"ID":"885425cc-20c2-4f74-b832-a5fe841098d0","Type":"ContainerStarted","Data":"f31d7dc7ea8ca32397909b288cf91795b1c13a8849895d0191c6d15d1b77106b"}
Nov 04 02:00:07.599630 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:07.599598    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52" event={"ID":"54cc3fcf-b55e-49f6-925b-3d9b62b94009","Type":"ContainerStarted","Data":"62402bafec5905602535fd1cd9dd22f514496e42d8e70c770cd14f4a0bdf4cef"}
Nov 04 02:00:09.446371 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:09.446171    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:15.693247 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:15.693205    4512 generic.go:356] "Generic (PLEG): container finished" podID="ef82cf23-0867-4c91-80d1-11fe242707a8" containerID="ee2eb8a32840bd456e43f9182e6493f0ae1760994cf8d0050f7a8f0d0081bddf" exitCode=0
Nov 04 02:00:15.693870 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:15.693354    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5" event={"ID":"ef82cf23-0867-4c91-80d1-11fe242707a8","Type":"ContainerDied","Data":"ee2eb8a32840bd456e43f9182e6493f0ae1760994cf8d0050f7a8f0d0081bddf"}
Nov 04 02:00:17.432064 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:17.432029    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:17.636346 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:17.636320    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:17.712812 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:17.712775    4512 generic.go:356] "Generic (PLEG): container finished" podID="2461de17-c237-444b-9d68-a36b63e2889e" containerID="392e0ae63227d15b723f465f900eec138c3c5262c3336c4814c65b3ea0b0981b" exitCode=0
Nov 04 02:00:17.712975 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:17.712882    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4" event={"ID":"2461de17-c237-444b-9d68-a36b63e2889e","Type":"ContainerDied","Data":"392e0ae63227d15b723f465f900eec138c3c5262c3336c4814c65b3ea0b0981b"}
Nov 04 02:00:17.719759 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:17.718928    4512 generic.go:356] "Generic (PLEG): container finished" podID="6452875f-0203-4b2e-8593-7e47cf9710d8" containerID="ae26b4be7df42b2c0aa29a8ec5b6a015c4bbbdfa746e71179f32f58d53b9a16e" exitCode=0
Nov 04 02:00:17.719759 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:17.719047    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774" event={"ID":"6452875f-0203-4b2e-8593-7e47cf9710d8","Type":"ContainerDied","Data":"ae26b4be7df42b2c0aa29a8ec5b6a015c4bbbdfa746e71179f32f58d53b9a16e"}
Nov 04 02:00:18.731739 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:18.731698    4512 generic.go:356] "Generic (PLEG): container finished" podID="d5de9d8b-4ca3-488a-a727-54f0204173d2" containerID="dffcb70119e9cdca40a313cb5226d93c002557cbe4c2a2e1dc4a50888a2908cb" exitCode=0
Nov 04 02:00:18.732208 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:18.731802    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj" event={"ID":"d5de9d8b-4ca3-488a-a727-54f0204173d2","Type":"ContainerDied","Data":"dffcb70119e9cdca40a313cb5226d93c002557cbe4c2a2e1dc4a50888a2908cb"}
Nov 04 02:00:19.062476 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:00:19.062433    4512 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885425cc_20c2_4f74_b832_a5fe841098d0.slice/crio-4f0077f0b53604f5635ade746e7b0e458b7d6600e7bbf7446886bf6dd88711d3.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 02:00:19.740100 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:19.739907    4512 generic.go:356] "Generic (PLEG): container finished" podID="885425cc-20c2-4f74-b832-a5fe841098d0" containerID="4f0077f0b53604f5635ade746e7b0e458b7d6600e7bbf7446886bf6dd88711d3" exitCode=0
Nov 04 02:00:19.740100 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:19.740041    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq" event={"ID":"885425cc-20c2-4f74-b832-a5fe841098d0","Type":"ContainerDied","Data":"4f0077f0b53604f5635ade746e7b0e458b7d6600e7bbf7446886bf6dd88711d3"}
Nov 04 02:00:19.744340 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:19.743790    4512 generic.go:356] "Generic (PLEG): container finished" podID="54cc3fcf-b55e-49f6-925b-3d9b62b94009" containerID="9eed2ff05c29643b5da1db7bc4225129714bf50788696a91c53d291c05e0c029" exitCode=0
Nov 04 02:00:19.744340 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:19.743923    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52" event={"ID":"54cc3fcf-b55e-49f6-925b-3d9b62b94009","Type":"ContainerDied","Data":"9eed2ff05c29643b5da1db7bc4225129714bf50788696a91c53d291c05e0c029"}
Nov 04 02:00:19.746573 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:19.746544    4512 generic.go:356] "Generic (PLEG): container finished" podID="8a0d7d21-bc96-46eb-8370-bdb0cd57620b" containerID="14c59f081252c3c1d1990259b72abf2eead71d1568a69f0d265740ffef403f0e" exitCode=0
Nov 04 02:00:19.746670 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:19.746598    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm" event={"ID":"8a0d7d21-bc96-46eb-8370-bdb0cd57620b","Type":"ContainerDied","Data":"14c59f081252c3c1d1990259b72abf2eead71d1568a69f0d265740ffef403f0e"}
Nov 04 02:00:20.486118 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.486079    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:00:20.486233 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.486163    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:00:20.486273 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.486230    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:00:20.598463 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.598005    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:00:20.598463 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.598091    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:00:20.598463 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.598129    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:00:20.773060 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.773032    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj" event={"ID":"d5de9d8b-4ca3-488a-a727-54f0204173d2","Type":"ContainerStarted","Data":"ffe08ba24120b75431f19242b6ce3da877f1cf0e9a478c345c2b80f8f4b7faab"}
Nov 04 02:00:20.779406 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.779383    4512 generic.go:356] "Generic (PLEG): container finished" podID="a6e91bb3-552f-4d57-ad3d-41c1b458e09f" containerID="e5aeacafc92bf9d7ef1f7203b85ce4489e5452c24c2785fe8d641db24d3903ab" exitCode=0
Nov 04 02:00:20.779488 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.779464    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c" event={"ID":"a6e91bb3-552f-4d57-ad3d-41c1b458e09f","Type":"ContainerDied","Data":"e5aeacafc92bf9d7ef1f7203b85ce4489e5452c24c2785fe8d641db24d3903ab"}
Nov 04 02:00:20.783181 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.783154    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829" event={"ID":"e2bbd606-cb7b-49c8-883d-aa3ae68d5d17","Type":"ContainerStarted","Data":"3456520fc9f3b50b26dff4266004b3d2fc8a2421f8e31bf411891339f7d72326"}
Nov 04 02:00:20.786812 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.786789    4512 generic.go:356] "Generic (PLEG): container finished" podID="a4405cc0-326d-4057-b87b-0b7c5efc10ae" containerID="256f66168e59172fecec4b6feff3613f8c5710d8cf7b767edcc0df97e0c7e748" exitCode=0
Nov 04 02:00:20.787406 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.787295    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq" event={"ID":"a4405cc0-326d-4057-b87b-0b7c5efc10ae","Type":"ContainerDied","Data":"256f66168e59172fecec4b6feff3613f8c5710d8cf7b767edcc0df97e0c7e748"}
Nov 04 02:00:20.796341 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.796317    4512 generic.go:356] "Generic (PLEG): container finished" podID="2461de17-c237-444b-9d68-a36b63e2889e" containerID="f72448cb67dc60b07edc0cd69fa65e4b17ec8c0088bfca32712506a41c6fc573" exitCode=0
Nov 04 02:00:20.796625 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:20.796380    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4" event={"ID":"2461de17-c237-444b-9d68-a36b63e2889e","Type":"ContainerDied","Data":"f72448cb67dc60b07edc0cd69fa65e4b17ec8c0088bfca32712506a41c6fc573"}
Nov 04 02:00:21.364384 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.364330    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj" podStartSLOduration=1.679385202 podStartE2EDuration="16.364315864s" totalImagesPullingTime="14.684930662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:21.363403765 +0000 UTC m=+283.168860397" watchObservedRunningTime="2026-11-04 02:00:21.364315864 +0000 UTC m=+283.169772495"
Nov 04 02:00:21.433703 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.433675    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:21.437730 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.437713    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:21.484123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.484075    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829" podStartSLOduration=2.160545475 podStartE2EDuration="16.484060101s" totalImagesPullingTime="14.323514626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:21.482793423 +0000 UTC m=+283.288250055" watchObservedRunningTime="2026-11-04 02:00:21.484060101 +0000 UTC m=+283.289516806"
Nov 04 02:00:21.751761 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.751735    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:21.757114 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.757091    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:21.804085 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.803990    4512 generic.go:356] "Generic (PLEG): container finished" podID="6452875f-0203-4b2e-8593-7e47cf9710d8" containerID="43817577ddead86e47ef7b1165b331a8d82be34a12532e3d6f2b9f12c9338881" exitCode=0
Nov 04 02:00:21.804085 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.804029    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774" event={"ID":"6452875f-0203-4b2e-8593-7e47cf9710d8","Type":"ContainerDied","Data":"43817577ddead86e47ef7b1165b331a8d82be34a12532e3d6f2b9f12c9338881"}
Nov 04 02:00:21.807081 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.807055    4512 generic.go:356] "Generic (PLEG): container finished" podID="d5de9d8b-4ca3-488a-a727-54f0204173d2" containerID="ffe08ba24120b75431f19242b6ce3da877f1cf0e9a478c345c2b80f8f4b7faab" exitCode=0
Nov 04 02:00:21.807169 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.807124    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj" event={"ID":"d5de9d8b-4ca3-488a-a727-54f0204173d2","Type":"ContainerDied","Data":"ffe08ba24120b75431f19242b6ce3da877f1cf0e9a478c345c2b80f8f4b7faab"}
Nov 04 02:00:21.809748 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.809720    4512 generic.go:356] "Generic (PLEG): container finished" podID="ef82cf23-0867-4c91-80d1-11fe242707a8" containerID="a616560289d9656cef53ee4d06b0c2981d139e8b37217b310c5a115f7f551fdf" exitCode=0
Nov 04 02:00:21.809845 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.809801    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5" event={"ID":"ef82cf23-0867-4c91-80d1-11fe242707a8","Type":"ContainerDied","Data":"a616560289d9656cef53ee4d06b0c2981d139e8b37217b310c5a115f7f551fdf"}
Nov 04 02:00:21.811984 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.811961    4512 generic.go:356] "Generic (PLEG): container finished" podID="8a0d7d21-bc96-46eb-8370-bdb0cd57620b" containerID="039b34c264f620dfbc4c60a0fd152da5828f6cd5d1c1c887b0757c6a78324c66" exitCode=0
Nov 04 02:00:21.812231 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.812212    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm" event={"ID":"8a0d7d21-bc96-46eb-8370-bdb0cd57620b","Type":"ContainerDied","Data":"039b34c264f620dfbc4c60a0fd152da5828f6cd5d1c1c887b0757c6a78324c66"}
Nov 04 02:00:21.814780 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.814761    4512 generic.go:356] "Generic (PLEG): container finished" podID="885425cc-20c2-4f74-b832-a5fe841098d0" containerID="72a8bc0522adee5943eae61551396fa19f5512cde21085b4df8af5c1603cfbbc" exitCode=0
Nov 04 02:00:21.814876 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.814849    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq" event={"ID":"885425cc-20c2-4f74-b832-a5fe841098d0","Type":"ContainerDied","Data":"72a8bc0522adee5943eae61551396fa19f5512cde21085b4df8af5c1603cfbbc"}
Nov 04 02:00:21.817003 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.816977    4512 generic.go:356] "Generic (PLEG): container finished" podID="c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" containerID="0e57f250f909ff2fe91d67dfe2a8e6aa00ab2357237e50f20d3df20124f3e2e8" exitCode=0
Nov 04 02:00:21.817483 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.817214    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb" event={"ID":"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b","Type":"ContainerDied","Data":"0e57f250f909ff2fe91d67dfe2a8e6aa00ab2357237e50f20d3df20124f3e2e8"}
Nov 04 02:00:21.819938 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.819913    4512 generic.go:356] "Generic (PLEG): container finished" podID="54cc3fcf-b55e-49f6-925b-3d9b62b94009" containerID="b151444b7ad8dbb56b57bc61af55540f988e73c3be6d9cd51bf2c01d08a33e2f" exitCode=0
Nov 04 02:00:21.820030 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.820011    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52" event={"ID":"54cc3fcf-b55e-49f6-925b-3d9b62b94009","Type":"ContainerDied","Data":"b151444b7ad8dbb56b57bc61af55540f988e73c3be6d9cd51bf2c01d08a33e2f"}
Nov 04 02:00:21.822004 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.821980    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj" event={"ID":"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2","Type":"ContainerStarted","Data":"c1545af460685b1fd70337ca46f94381594694c7f0115d7f890eb7a1284cfcb9"}
Nov 04 02:00:21.822197 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.822173    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:21.823121 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.822367    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:21.823218 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.823197    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829"
Nov 04 02:00:21.823326 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:21.823310    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:00:22.834294 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:22.834259    4512 generic.go:356] "Generic (PLEG): container finished" podID="a4405cc0-326d-4057-b87b-0b7c5efc10ae" containerID="a6327a139824b545849888168c82c88f6695681cd28a090ff95747d0b008a8eb" exitCode=0
Nov 04 02:00:22.834758 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:22.834462    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq" event={"ID":"a4405cc0-326d-4057-b87b-0b7c5efc10ae","Type":"ContainerDied","Data":"a6327a139824b545849888168c82c88f6695681cd28a090ff95747d0b008a8eb"}
Nov 04 02:00:22.846439 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:22.846412    4512 generic.go:356] "Generic (PLEG): container finished" podID="a6e91bb3-552f-4d57-ad3d-41c1b458e09f" containerID="c4c1223f28c88d07f4d3750ca58c886a4c98c88ceceadc0b5e1d2257ea27169a" exitCode=0
Nov 04 02:00:22.847649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:22.847627    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c" event={"ID":"a6e91bb3-552f-4d57-ad3d-41c1b458e09f","Type":"ContainerDied","Data":"c4c1223f28c88d07f4d3750ca58c886a4c98c88ceceadc0b5e1d2257ea27169a"}
Nov 04 02:00:25.483557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:25.483481    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:26.858062 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:00:26.857797    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:00:29.954088 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:29.954045    4512 generic.go:356] "Generic (PLEG): container finished" podID="c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" containerID="da31ccff7665639a5cac1f56d979ee21979eb481a43f691650f710f6c95b899b" exitCode=0
Nov 04 02:00:29.954593 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:29.954159    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb" event={"ID":"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b","Type":"ContainerDied","Data":"da31ccff7665639a5cac1f56d979ee21979eb481a43f691650f710f6c95b899b"}
Nov 04 02:00:40.074279 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:40.073345    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4" event={"ID":"2461de17-c237-444b-9d68-a36b63e2889e","Type":"ContainerStarted","Data":"d9133f5dc77212370805603f8d0d3d3141143f8a9d9f730e444e99b3898f36b3"}
Nov 04 02:00:41.087057 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:41.086617    4512 generic.go:356] "Generic (PLEG): container finished" podID="2461de17-c237-444b-9d68-a36b63e2889e" containerID="d9133f5dc77212370805603f8d0d3d3141143f8a9d9f730e444e99b3898f36b3" exitCode=0
Nov 04 02:00:41.087057 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:41.086685    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4" event={"ID":"2461de17-c237-444b-9d68-a36b63e2889e","Type":"ContainerDied","Data":"d9133f5dc77212370805603f8d0d3d3141143f8a9d9f730e444e99b3898f36b3"}
Nov 04 02:00:42.339199 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:42.339171    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:42.411092 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:42.410322    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-bundle\" (UniqueName: \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-bundle\") pod \"2461de17-c237-444b-9d68-a36b63e2889e\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") "
Nov 04 02:00:42.411092 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:42.410361    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2461de17-c237-444b-9d68-a36b63e2889e-kube-api-access-k7phc\" (UniqueName: \"kubernetes.io/projected/2461de17-c237-444b-9d68-a36b63e2889e-kube-api-access-k7phc\") pod \"2461de17-c237-444b-9d68-a36b63e2889e\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") "
Nov 04 02:00:42.411092 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:42.410419    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-util\" (UniqueName: \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-util\") pod \"2461de17-c237-444b-9d68-a36b63e2889e\" (UID: \"2461de17-c237-444b-9d68-a36b63e2889e\") "
Nov 04 02:00:42.413075 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:42.413013    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-bundle" pod "2461de17-c237-444b-9d68-a36b63e2889e" (UID: "2461de17-c237-444b-9d68-a36b63e2889e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:42.415127 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:42.415084    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2461de17-c237-444b-9d68-a36b63e2889e-kube-api-access-k7phc" pod "2461de17-c237-444b-9d68-a36b63e2889e" (UID: "2461de17-c237-444b-9d68-a36b63e2889e"). InnerVolumeSpecName "kube-api-access-k7phc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:42.416974 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:42.416948    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-util" pod "2461de17-c237-444b-9d68-a36b63e2889e" (UID: "2461de17-c237-444b-9d68-a36b63e2889e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:42.512072 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:42.512032    4512 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:42.512243 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:42.512077    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7phc\" (UniqueName: \"kubernetes.io/projected/2461de17-c237-444b-9d68-a36b63e2889e-kube-api-access-k7phc\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:42.512243 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:42.512094    4512 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2461de17-c237-444b-9d68-a36b63e2889e-util\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:43.103330 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.103301    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4"
Nov 04 02:00:43.103576 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.103297    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719m6j4" event={"ID":"2461de17-c237-444b-9d68-a36b63e2889e","Type":"ContainerDied","Data":"58cd900bdc8fa5f19e57481f1881314df9db853d7597383f4a09cac949ac4072"}
Nov 04 02:00:43.103576 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.103448    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58cd900bdc8fa5f19e57481f1881314df9db853d7597383f4a09cac949ac4072"
Nov 04 02:00:43.109713 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.109686    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774" event={"ID":"6452875f-0203-4b2e-8593-7e47cf9710d8","Type":"ContainerStarted","Data":"38aac03fda23b8d7e19d32d0dfa7383e3bd05eab64c53a3a5d7f955e27f92e57"}
Nov 04 02:00:43.112074 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.112049    4512 generic.go:356] "Generic (PLEG): container finished" podID="8a0d7d21-bc96-46eb-8370-bdb0cd57620b" containerID="8644637fb3b9033ed8e90953036b94af58b2f0e942d69a7a4800d61845d6a8ff" exitCode=0
Nov 04 02:00:43.112231 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.112204    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm" event={"ID":"8a0d7d21-bc96-46eb-8370-bdb0cd57620b","Type":"ContainerDied","Data":"8644637fb3b9033ed8e90953036b94af58b2f0e942d69a7a4800d61845d6a8ff"}
Nov 04 02:00:43.114896 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.114873    4512 generic.go:356] "Generic (PLEG): container finished" podID="885425cc-20c2-4f74-b832-a5fe841098d0" containerID="9c7ed88f118cf1885c859d8b3c298b618e4150e0383e177a162397e7facc3136" exitCode=0
Nov 04 02:00:43.114993 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.114944    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq" event={"ID":"885425cc-20c2-4f74-b832-a5fe841098d0","Type":"ContainerDied","Data":"9c7ed88f118cf1885c859d8b3c298b618e4150e0383e177a162397e7facc3136"}
Nov 04 02:00:43.117323 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.117300    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb" event={"ID":"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b","Type":"ContainerStarted","Data":"9bfb72060b5c8018f9e07f7695451cba35265f6fe2f5f653099be529c8246b5f"}
Nov 04 02:00:43.227200 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.227149    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774" podStartSLOduration=3.7447914989999997 podStartE2EDuration="44.227131693s" totalImagesPullingTime="40.482340194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:43.225901333 +0000 UTC m=+305.031357964" watchObservedRunningTime="2026-11-04 02:00:43.227131693 +0000 UTC m=+305.032588325"
Nov 04 02:00:43.516252 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:43.516207    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb" podStartSLOduration=4.225081203 podStartE2EDuration="44.516194454s" totalImagesPullingTime="40.291113251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:43.513768668 +0000 UTC m=+305.319225302" watchObservedRunningTime="2026-11-04 02:00:43.516194454 +0000 UTC m=+305.321651084"
Nov 04 02:00:44.124534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.124488    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq" event={"ID":"a4405cc0-326d-4057-b87b-0b7c5efc10ae","Type":"ContainerStarted","Data":"1bb3e15bbdfce46822f246fba1c82744aec062324e76f27ba089252adefa5e13"}
Nov 04 02:00:44.126431 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.126404    4512 generic.go:356] "Generic (PLEG): container finished" podID="54cc3fcf-b55e-49f6-925b-3d9b62b94009" containerID="ae290ae28bdad4d207194f89de2a767b9291cc6ae25fb5865fc47fb23f4fd96f" exitCode=0
Nov 04 02:00:44.126544 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.126477    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52" event={"ID":"54cc3fcf-b55e-49f6-925b-3d9b62b94009","Type":"ContainerDied","Data":"ae290ae28bdad4d207194f89de2a767b9291cc6ae25fb5865fc47fb23f4fd96f"}
Nov 04 02:00:44.128557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.128493    4512 generic.go:356] "Generic (PLEG): container finished" podID="6452875f-0203-4b2e-8593-7e47cf9710d8" containerID="38aac03fda23b8d7e19d32d0dfa7383e3bd05eab64c53a3a5d7f955e27f92e57" exitCode=0
Nov 04 02:00:44.128557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.128541    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774" event={"ID":"6452875f-0203-4b2e-8593-7e47cf9710d8","Type":"ContainerDied","Data":"38aac03fda23b8d7e19d32d0dfa7383e3bd05eab64c53a3a5d7f955e27f92e57"}
Nov 04 02:00:44.130151 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.130135    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:00:44.130764 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.130744    4512 generic.go:356] "Generic (PLEG): container finished" podID="d5de9d8b-4ca3-488a-a727-54f0204173d2" containerID="be0d6cdbbb085273c7615769d1e693e5ef26caa27dd88f2703db7eae87ac5023" exitCode=1
Nov 04 02:00:44.130852 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.130824    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj" event={"ID":"d5de9d8b-4ca3-488a-a727-54f0204173d2","Type":"ContainerDied","Data":"be0d6cdbbb085273c7615769d1e693e5ef26caa27dd88f2703db7eae87ac5023"}
Nov 04 02:00:44.132913 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.132894    4512 generic.go:356] "Generic (PLEG): container finished" podID="ef82cf23-0867-4c91-80d1-11fe242707a8" containerID="8d80a76bfedea8f3a16c6cea1ccd595ade576d07cebd52c2efaf50e09077fe88" exitCode=0
Nov 04 02:00:44.133005 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.132974    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5" event={"ID":"ef82cf23-0867-4c91-80d1-11fe242707a8","Type":"ContainerDied","Data":"8d80a76bfedea8f3a16c6cea1ccd595ade576d07cebd52c2efaf50e09077fe88"}
Nov 04 02:00:44.134881 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.134863    4512 generic.go:356] "Generic (PLEG): container finished" podID="a6e91bb3-552f-4d57-ad3d-41c1b458e09f" containerID="c3c91c47e1dda8afac8a49b0b1948109a8ef6177c5ba624b9e1d00aee309b001" exitCode=0
Nov 04 02:00:44.134973 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.134889    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c" event={"ID":"a6e91bb3-552f-4d57-ad3d-41c1b458e09f","Type":"ContainerDied","Data":"c3c91c47e1dda8afac8a49b0b1948109a8ef6177c5ba624b9e1d00aee309b001"}
Nov 04 02:00:44.136905 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.136888    4512 generic.go:356] "Generic (PLEG): container finished" podID="c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" containerID="9bfb72060b5c8018f9e07f7695451cba35265f6fe2f5f653099be529c8246b5f" exitCode=0
Nov 04 02:00:44.136975 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.136954    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb" event={"ID":"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b","Type":"ContainerDied","Data":"9bfb72060b5c8018f9e07f7695451cba35265f6fe2f5f653099be529c8246b5f"}
Nov 04 02:00:44.244305 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.244250    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq" podStartSLOduration=2.686294715 podStartE2EDuration="45.244235428s" totalImagesPullingTime="42.557940713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:44.24374757 +0000 UTC m=+306.049204202" watchObservedRunningTime="2026-11-04 02:00:44.244235428 +0000 UTC m=+306.049692058"
Nov 04 02:00:44.269214 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.269195    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:44.295824 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.295801    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:44.326125 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.326103    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-bundle\") pod \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") "
Nov 04 02:00:44.326219 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.326193    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-util\" (UniqueName: \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-util\") pod \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") "
Nov 04 02:00:44.326219 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.326214    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-kube-api-access-m4kq2\" (UniqueName: \"kubernetes.io/projected/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-kube-api-access-m4kq2\") pod \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\" (UID: \"8a0d7d21-bc96-46eb-8370-bdb0cd57620b\") "
Nov 04 02:00:44.326606 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.326583    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-bundle" pod "8a0d7d21-bc96-46eb-8370-bdb0cd57620b" (UID: "8a0d7d21-bc96-46eb-8370-bdb0cd57620b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:44.328282 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.328261    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-kube-api-access-m4kq2" pod "8a0d7d21-bc96-46eb-8370-bdb0cd57620b" (UID: "8a0d7d21-bc96-46eb-8370-bdb0cd57620b"). InnerVolumeSpecName "kube-api-access-m4kq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:44.331427 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.331408    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-util" pod "8a0d7d21-bc96-46eb-8370-bdb0cd57620b" (UID: "8a0d7d21-bc96-46eb-8370-bdb0cd57620b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:44.427158 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.427137    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-bundle\" (UniqueName: \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-bundle\") pod \"885425cc-20c2-4f74-b832-a5fe841098d0\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") "
Nov 04 02:00:44.427238 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.427195    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/885425cc-20c2-4f74-b832-a5fe841098d0-kube-api-access-9bss8\" (UniqueName: \"kubernetes.io/projected/885425cc-20c2-4f74-b832-a5fe841098d0-kube-api-access-9bss8\") pod \"885425cc-20c2-4f74-b832-a5fe841098d0\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") "
Nov 04 02:00:44.427345 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.427330    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-util\" (UniqueName: \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-util\") pod \"885425cc-20c2-4f74-b832-a5fe841098d0\" (UID: \"885425cc-20c2-4f74-b832-a5fe841098d0\") "
Nov 04 02:00:44.427581 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.427558    4512 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-util\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:44.427673 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.427584    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4kq2\" (UniqueName: \"kubernetes.io/projected/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-kube-api-access-m4kq2\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:44.427673 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.427597    4512 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a0d7d21-bc96-46eb-8370-bdb0cd57620b-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:44.427673 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.427635    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-bundle" pod "885425cc-20c2-4f74-b832-a5fe841098d0" (UID: "885425cc-20c2-4f74-b832-a5fe841098d0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:44.429107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.429092    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/885425cc-20c2-4f74-b832-a5fe841098d0-kube-api-access-9bss8" pod "885425cc-20c2-4f74-b832-a5fe841098d0" (UID: "885425cc-20c2-4f74-b832-a5fe841098d0"). InnerVolumeSpecName "kube-api-access-9bss8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:44.432294 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.432274    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-util" pod "885425cc-20c2-4f74-b832-a5fe841098d0" (UID: "885425cc-20c2-4f74-b832-a5fe841098d0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:44.528615 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.528592    4512 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-util\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:44.528615 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.528612    4512 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/885425cc-20c2-4f74-b832-a5fe841098d0-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:44.528927 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:44.528621    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bss8\" (UniqueName: \"kubernetes.io/projected/885425cc-20c2-4f74-b832-a5fe841098d0-kube-api-access-9bss8\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.144277 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.144247    4512 generic.go:356] "Generic (PLEG): container finished" podID="a4405cc0-326d-4057-b87b-0b7c5efc10ae" containerID="1bb3e15bbdfce46822f246fba1c82744aec062324e76f27ba089252adefa5e13" exitCode=0
Nov 04 02:00:45.144433 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.144355    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq" event={"ID":"a4405cc0-326d-4057-b87b-0b7c5efc10ae","Type":"ContainerDied","Data":"1bb3e15bbdfce46822f246fba1c82744aec062324e76f27ba089252adefa5e13"}
Nov 04 02:00:45.146331 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.146313    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm"
Nov 04 02:00:45.146438 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.146312    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7ngqgm" event={"ID":"8a0d7d21-bc96-46eb-8370-bdb0cd57620b","Type":"ContainerDied","Data":"9e0a435224230429dcd005251e3b5d4ca3e8082f3ef21254e44d2b9776fbc40e"}
Nov 04 02:00:45.146438 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.146412    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e0a435224230429dcd005251e3b5d4ca3e8082f3ef21254e44d2b9776fbc40e"
Nov 04 02:00:45.148303 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.148283    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq"
Nov 04 02:00:45.148407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.148317    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6scjsq" event={"ID":"885425cc-20c2-4f74-b832-a5fe841098d0","Type":"ContainerDied","Data":"f31d7dc7ea8ca32397909b288cf91795b1c13a8849895d0191c6d15d1b77106b"}
Nov 04 02:00:45.148407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.148352    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f31d7dc7ea8ca32397909b288cf91795b1c13a8849895d0191c6d15d1b77106b"
Nov 04 02:00:45.334965 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.334937    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 02:00:45.377394 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.377372    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 02:00:45.381191 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.381169    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 02:00:45.385029 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.385010    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:45.389649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.389632    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 02:00:45.406163 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.406146    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:00:45.407865 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.407805    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 02:00:45.436451 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436415    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-util\" (UniqueName: \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-util\") pod \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") "
Nov 04 02:00:45.436574 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436477    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-util\" (UniqueName: \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-util\") pod \"ef82cf23-0867-4c91-80d1-11fe242707a8\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") "
Nov 04 02:00:45.436574 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436501    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-bundle\") pod \"ef82cf23-0867-4c91-80d1-11fe242707a8\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") "
Nov 04 02:00:45.436574 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436540    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-util\" (UniqueName: \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-util\") pod \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") "
Nov 04 02:00:45.436574 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436560    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-util\" (UniqueName: \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-util\") pod \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") "
Nov 04 02:00:45.436777 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436586    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-kube-api-access-7fmrl\" (UniqueName: \"kubernetes.io/projected/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-kube-api-access-7fmrl\") pod \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") "
Nov 04 02:00:45.436777 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436614    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-bundle\") pod \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") "
Nov 04 02:00:45.436777 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436644    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-bundle\" (UniqueName: \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-bundle\") pod \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") "
Nov 04 02:00:45.436777 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436681    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-util\" (UniqueName: \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-util\") pod \"6452875f-0203-4b2e-8593-7e47cf9710d8\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") "
Nov 04 02:00:45.436777 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436722    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-bundle\") pod \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\" (UID: \"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b\") "
Nov 04 02:00:45.436777 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436759    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6452875f-0203-4b2e-8593-7e47cf9710d8-kube-api-access-qvsp4\" (UniqueName: \"kubernetes.io/projected/6452875f-0203-4b2e-8593-7e47cf9710d8-kube-api-access-qvsp4\") pod \"6452875f-0203-4b2e-8593-7e47cf9710d8\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") "
Nov 04 02:00:45.437054 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436787    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54cc3fcf-b55e-49f6-925b-3d9b62b94009-kube-api-access-9kd2w\" (UniqueName: \"kubernetes.io/projected/54cc3fcf-b55e-49f6-925b-3d9b62b94009-kube-api-access-9kd2w\") pod \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\" (UID: \"54cc3fcf-b55e-49f6-925b-3d9b62b94009\") "
Nov 04 02:00:45.437054 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436820    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef82cf23-0867-4c91-80d1-11fe242707a8-kube-api-access-44nh5\" (UniqueName: \"kubernetes.io/projected/ef82cf23-0867-4c91-80d1-11fe242707a8-kube-api-access-44nh5\") pod \"ef82cf23-0867-4c91-80d1-11fe242707a8\" (UID: \"ef82cf23-0867-4c91-80d1-11fe242707a8\") "
Nov 04 02:00:45.437054 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436851    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-bundle\" (UniqueName: \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-bundle\") pod \"6452875f-0203-4b2e-8593-7e47cf9710d8\" (UID: \"6452875f-0203-4b2e-8593-7e47cf9710d8\") "
Nov 04 02:00:45.437054 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.436894    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-kube-api-access-p2w5n\" (UniqueName: \"kubernetes.io/projected/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-kube-api-access-p2w5n\") pod \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\" (UID: \"a6e91bb3-552f-4d57-ad3d-41c1b458e09f\") "
Nov 04 02:00:45.437235 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.437084    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-bundle" pod "a6e91bb3-552f-4d57-ad3d-41c1b458e09f" (UID: "a6e91bb3-552f-4d57-ad3d-41c1b458e09f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.437235 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.437213    4512 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.437339 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.437312    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-bundle" pod "54cc3fcf-b55e-49f6-925b-3d9b62b94009" (UID: "54cc3fcf-b55e-49f6-925b-3d9b62b94009"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.437625 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.437604    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-bundle" pod "ef82cf23-0867-4c91-80d1-11fe242707a8" (UID: "ef82cf23-0867-4c91-80d1-11fe242707a8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.438689 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.438274    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-bundle" pod "c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" (UID: "c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.439284 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.439059    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-bundle" pod "6452875f-0203-4b2e-8593-7e47cf9710d8" (UID: "6452875f-0203-4b2e-8593-7e47cf9710d8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.439284 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.439156    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-kube-api-access-7fmrl" pod "c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" (UID: "c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b"). InnerVolumeSpecName "kube-api-access-7fmrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:45.440141 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.440116    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54cc3fcf-b55e-49f6-925b-3d9b62b94009-kube-api-access-9kd2w" pod "54cc3fcf-b55e-49f6-925b-3d9b62b94009" (UID: "54cc3fcf-b55e-49f6-925b-3d9b62b94009"). InnerVolumeSpecName "kube-api-access-9kd2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:45.441318 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.441291    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6452875f-0203-4b2e-8593-7e47cf9710d8-kube-api-access-qvsp4" pod "6452875f-0203-4b2e-8593-7e47cf9710d8" (UID: "6452875f-0203-4b2e-8593-7e47cf9710d8"). InnerVolumeSpecName "kube-api-access-qvsp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:45.442019 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.441990    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef82cf23-0867-4c91-80d1-11fe242707a8-kube-api-access-44nh5" pod "ef82cf23-0867-4c91-80d1-11fe242707a8" (UID: "ef82cf23-0867-4c91-80d1-11fe242707a8"). InnerVolumeSpecName "kube-api-access-44nh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:45.442340 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.442319    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-kube-api-access-p2w5n" pod "a6e91bb3-552f-4d57-ad3d-41c1b458e09f" (UID: "a6e91bb3-552f-4d57-ad3d-41c1b458e09f"). InnerVolumeSpecName "kube-api-access-p2w5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:45.446894 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.446870    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-util" pod "c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" (UID: "c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.446979 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.446947    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-util" pod "ef82cf23-0867-4c91-80d1-11fe242707a8" (UID: "ef82cf23-0867-4c91-80d1-11fe242707a8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.447070 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.447050    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-util" pod "6452875f-0203-4b2e-8593-7e47cf9710d8" (UID: "6452875f-0203-4b2e-8593-7e47cf9710d8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.447206 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.447191    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-util" pod "a6e91bb3-552f-4d57-ad3d-41c1b458e09f" (UID: "a6e91bb3-552f-4d57-ad3d-41c1b458e09f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.447649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.447630    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-util" pod "54cc3fcf-b55e-49f6-925b-3d9b62b94009" (UID: "54cc3fcf-b55e-49f6-925b-3d9b62b94009"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.537583 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537560    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5de9d8b-4ca3-488a-a727-54f0204173d2-kube-api-access-gzj8t\" (UniqueName: \"kubernetes.io/projected/d5de9d8b-4ca3-488a-a727-54f0204173d2-kube-api-access-gzj8t\") pod \"d5de9d8b-4ca3-488a-a727-54f0204173d2\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") "
Nov 04 02:00:45.537891 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537620    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-bundle\") pod \"d5de9d8b-4ca3-488a-a727-54f0204173d2\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") "
Nov 04 02:00:45.537891 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537664    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-util\" (UniqueName: \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-util\") pod \"d5de9d8b-4ca3-488a-a727-54f0204173d2\" (UID: \"d5de9d8b-4ca3-488a-a727-54f0204173d2\") "
Nov 04 02:00:45.537891 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537832    4512 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.537891 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537851    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvsp4\" (UniqueName: \"kubernetes.io/projected/6452875f-0203-4b2e-8593-7e47cf9710d8-kube-api-access-qvsp4\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.537891 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537865    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kd2w\" (UniqueName: \"kubernetes.io/projected/54cc3fcf-b55e-49f6-925b-3d9b62b94009-kube-api-access-9kd2w\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.537891 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537879    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44nh5\" (UniqueName: \"kubernetes.io/projected/ef82cf23-0867-4c91-80d1-11fe242707a8-kube-api-access-44nh5\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537893    4512 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537905    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2w5n\" (UniqueName: \"kubernetes.io/projected/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-kube-api-access-p2w5n\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537918    4512 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-util\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537931    4512 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-util\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537945    4512 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef82cf23-0867-4c91-80d1-11fe242707a8-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537957    4512 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6e91bb3-552f-4d57-ad3d-41c1b458e09f-util\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537969    4512 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-util\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537981    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fmrl\" (UniqueName: \"kubernetes.io/projected/c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b-kube-api-access-7fmrl\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.537992    4512 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54cc3fcf-b55e-49f6-925b-3d9b62b94009-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.538004    4512 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6452875f-0203-4b2e-8593-7e47cf9710d8-util\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.538403 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.538196    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-bundle" pod "d5de9d8b-4ca3-488a-a727-54f0204173d2" (UID: "d5de9d8b-4ca3-488a-a727-54f0204173d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.539451 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.539435    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5de9d8b-4ca3-488a-a727-54f0204173d2-kube-api-access-gzj8t" pod "d5de9d8b-4ca3-488a-a727-54f0204173d2" (UID: "d5de9d8b-4ca3-488a-a727-54f0204173d2"). InnerVolumeSpecName "kube-api-access-gzj8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:45.544591 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.544561    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-util" pod "d5de9d8b-4ca3-488a-a727-54f0204173d2" (UID: "d5de9d8b-4ca3-488a-a727-54f0204173d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:45.639091 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.639066    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzj8t\" (UniqueName: \"kubernetes.io/projected/d5de9d8b-4ca3-488a-a727-54f0204173d2-kube-api-access-gzj8t\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.639091 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.639091    4512 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:45.639234 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:45.639101    4512 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5de9d8b-4ca3-488a-a727-54f0204173d2-util\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:46.159064 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.159034    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774" event={"ID":"6452875f-0203-4b2e-8593-7e47cf9710d8","Type":"ContainerDied","Data":"a709f90dafe47d01c6c893b6036054314779d09db0eab2393e6b250d7b27aefd"}
Nov 04 02:00:46.159215 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.159101    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7v2774"
Nov 04 02:00:46.159215 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.159106    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a709f90dafe47d01c6c893b6036054314779d09db0eab2393e6b250d7b27aefd"
Nov 04 02:00:46.160690 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.160672    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:00:46.161397 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.161373    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj" event={"ID":"d5de9d8b-4ca3-488a-a727-54f0204173d2","Type":"ContainerDied","Data":"108d861fefd733e4e4e85c299b6685537d34a860a1e456c50f95218eac8d26cc"}
Nov 04 02:00:46.161521 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.161423    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="108d861fefd733e4e4e85c299b6685537d34a860a1e456c50f95218eac8d26cc"
Nov 04 02:00:46.161521 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.161388    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj"
Nov 04 02:00:46.163684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.163667    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5"
Nov 04 02:00:46.163771 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.163665    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gw6p5" event={"ID":"ef82cf23-0867-4c91-80d1-11fe242707a8","Type":"ContainerDied","Data":"5c8298a22c6a4a2c03817a19bf9d7c4581c1f9ee2ff0afc889f1094baf69650b"}
Nov 04 02:00:46.163842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.163785    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c8298a22c6a4a2c03817a19bf9d7c4581c1f9ee2ff0afc889f1094baf69650b"
Nov 04 02:00:46.165722 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.165697    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c" event={"ID":"a6e91bb3-552f-4d57-ad3d-41c1b458e09f","Type":"ContainerDied","Data":"55c3eef83c3255539b10686813bef9d74bb4d8d60e9eb5b0b1ebfe9f4d717942"}
Nov 04 02:00:46.165804 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.165739    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55c3eef83c3255539b10686813bef9d74bb4d8d60e9eb5b0b1ebfe9f4d717942"
Nov 04 02:00:46.165804 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.165706    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6rns8c"
Nov 04 02:00:46.170361 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.170344    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb"
Nov 04 02:00:46.170450 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.170415    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c8bjrb" event={"ID":"c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b","Type":"ContainerDied","Data":"df0ee1a64ee5962f53233ffb0ba0b20cbcda69c63cb4c4073782c91da3b14ca3"}
Nov 04 02:00:46.170543 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.170452    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df0ee1a64ee5962f53233ffb0ba0b20cbcda69c63cb4c4073782c91da3b14ca3"
Nov 04 02:00:46.174279 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.174074    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52"
Nov 04 02:00:46.174279 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.174090    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at9v52" event={"ID":"54cc3fcf-b55e-49f6-925b-3d9b62b94009","Type":"ContainerDied","Data":"62402bafec5905602535fd1cd9dd22f514496e42d8e70c770cd14f4a0bdf4cef"}
Nov 04 02:00:46.174279 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.174125    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62402bafec5905602535fd1cd9dd22f514496e42d8e70c770cd14f4a0bdf4cef"
Nov 04 02:00:46.252236 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.252207    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:46.305843 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.305823    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 02:00:46.346090 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.346067    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4405cc0-326d-4057-b87b-0b7c5efc10ae-kube-api-access-9tljq\" (UniqueName: \"kubernetes.io/projected/a4405cc0-326d-4057-b87b-0b7c5efc10ae-kube-api-access-9tljq\") pod \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") "
Nov 04 02:00:46.346202 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.346125    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-bundle\") pod \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") "
Nov 04 02:00:46.346256 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.346230    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-util\" (UniqueName: \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-util\") pod \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\" (UID: \"a4405cc0-326d-4057-b87b-0b7c5efc10ae\") "
Nov 04 02:00:46.346705 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.346683    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-bundle" pod "a4405cc0-326d-4057-b87b-0b7c5efc10ae" (UID: "a4405cc0-326d-4057-b87b-0b7c5efc10ae"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:46.348124 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.348099    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4405cc0-326d-4057-b87b-0b7c5efc10ae-kube-api-access-9tljq" pod "a4405cc0-326d-4057-b87b-0b7c5efc10ae" (UID: "a4405cc0-326d-4057-b87b-0b7c5efc10ae"). InnerVolumeSpecName "kube-api-access-9tljq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:46.351754 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.351726    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-util" pod "a4405cc0-326d-4057-b87b-0b7c5efc10ae" (UID: "a4405cc0-326d-4057-b87b-0b7c5efc10ae"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:46.447888 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.447836    4512 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-util\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:46.447888 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.447857    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tljq\" (UniqueName: \"kubernetes.io/projected/a4405cc0-326d-4057-b87b-0b7c5efc10ae-kube-api-access-9tljq\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:46.447888 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.447872    4512 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4405cc0-326d-4057-b87b-0b7c5efc10ae-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:46.483626 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.483596    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:46.883269 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.883235    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:46.943946 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.943923    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"]
Nov 04 02:00:46.944242 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944230    4512 cpu_manager.go:414] "containerMap: removing container" podUID="2461de17-c237-444b-9d68-a36b63e2889e" containerName="pull"
Nov 04 02:00:46.944290 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944242    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2461de17-c237-444b-9d68-a36b63e2889e" containerName="pull"
Nov 04 02:00:46.944290 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944252    4512 cpu_manager.go:414] "containerMap: removing container" podUID="a4405cc0-326d-4057-b87b-0b7c5efc10ae" containerName="pull"
Nov 04 02:00:46.944290 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944257    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4405cc0-326d-4057-b87b-0b7c5efc10ae" containerName="pull"
Nov 04 02:00:46.944290 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944267    4512 cpu_manager.go:414] "containerMap: removing container" podUID="885425cc-20c2-4f74-b832-a5fe841098d0" containerName="extract"
Nov 04 02:00:46.944290 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944272    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="885425cc-20c2-4f74-b832-a5fe841098d0" containerName="extract"
Nov 04 02:00:46.944290 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944283    4512 cpu_manager.go:414] "containerMap: removing container" podUID="2461de17-c237-444b-9d68-a36b63e2889e" containerName="util"
Nov 04 02:00:46.944290 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944289    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2461de17-c237-444b-9d68-a36b63e2889e" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944295    4512 cpu_manager.go:414] "containerMap: removing container" podUID="885425cc-20c2-4f74-b832-a5fe841098d0" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944300    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="885425cc-20c2-4f74-b832-a5fe841098d0" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944308    4512 cpu_manager.go:414] "containerMap: removing container" podUID="c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944312    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944317    4512 cpu_manager.go:414] "containerMap: removing container" podUID="8a0d7d21-bc96-46eb-8370-bdb0cd57620b" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944322    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a0d7d21-bc96-46eb-8370-bdb0cd57620b" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944341    4512 cpu_manager.go:414] "containerMap: removing container" podUID="54cc3fcf-b55e-49f6-925b-3d9b62b94009" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944346    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54cc3fcf-b55e-49f6-925b-3d9b62b94009" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944352    4512 cpu_manager.go:414] "containerMap: removing container" podUID="d5de9d8b-4ca3-488a-a727-54f0204173d2" containerName="pull"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944357    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5de9d8b-4ca3-488a-a727-54f0204173d2" containerName="pull"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944368    4512 cpu_manager.go:414] "containerMap: removing container" podUID="54cc3fcf-b55e-49f6-925b-3d9b62b94009" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944372    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54cc3fcf-b55e-49f6-925b-3d9b62b94009" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944378    4512 cpu_manager.go:414] "containerMap: removing container" podUID="8a0d7d21-bc96-46eb-8370-bdb0cd57620b" containerName="pull"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944382    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a0d7d21-bc96-46eb-8370-bdb0cd57620b" containerName="pull"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944387    4512 cpu_manager.go:414] "containerMap: removing container" podUID="ef82cf23-0867-4c91-80d1-11fe242707a8" containerName="pull"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944391    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef82cf23-0867-4c91-80d1-11fe242707a8" containerName="pull"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944399    4512 cpu_manager.go:414] "containerMap: removing container" podUID="c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944403    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944410    4512 cpu_manager.go:414] "containerMap: removing container" podUID="d5de9d8b-4ca3-488a-a727-54f0204173d2" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944414    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5de9d8b-4ca3-488a-a727-54f0204173d2" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944419    4512 cpu_manager.go:414] "containerMap: removing container" podUID="ef82cf23-0867-4c91-80d1-11fe242707a8" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944423    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef82cf23-0867-4c91-80d1-11fe242707a8" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944428    4512 cpu_manager.go:414] "containerMap: removing container" podUID="d5de9d8b-4ca3-488a-a727-54f0204173d2" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944433    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5de9d8b-4ca3-488a-a727-54f0204173d2" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944438    4512 cpu_manager.go:414] "containerMap: removing container" podUID="a4405cc0-326d-4057-b87b-0b7c5efc10ae" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944442    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4405cc0-326d-4057-b87b-0b7c5efc10ae" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944450    4512 cpu_manager.go:414] "containerMap: removing container" podUID="6452875f-0203-4b2e-8593-7e47cf9710d8" containerName="pull"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944455    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6452875f-0203-4b2e-8593-7e47cf9710d8" containerName="pull"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944461    4512 cpu_manager.go:414] "containerMap: removing container" podUID="ef82cf23-0867-4c91-80d1-11fe242707a8" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944465    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef82cf23-0867-4c91-80d1-11fe242707a8" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944472    4512 cpu_manager.go:414] "containerMap: removing container" podUID="8a0d7d21-bc96-46eb-8370-bdb0cd57620b" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944476    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a0d7d21-bc96-46eb-8370-bdb0cd57620b" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944482    4512 cpu_manager.go:414] "containerMap: removing container" podUID="6452875f-0203-4b2e-8593-7e47cf9710d8" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944486    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6452875f-0203-4b2e-8593-7e47cf9710d8" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944492    4512 cpu_manager.go:414] "containerMap: removing container" podUID="a6e91bb3-552f-4d57-ad3d-41c1b458e09f" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944496    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6e91bb3-552f-4d57-ad3d-41c1b458e09f" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944516    4512 cpu_manager.go:414] "containerMap: removing container" podUID="2461de17-c237-444b-9d68-a36b63e2889e" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944521    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2461de17-c237-444b-9d68-a36b63e2889e" containerName="extract"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944528    4512 cpu_manager.go:414] "containerMap: removing container" podUID="a6e91bb3-552f-4d57-ad3d-41c1b458e09f" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944533    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6e91bb3-552f-4d57-ad3d-41c1b458e09f" containerName="util"
Nov 04 02:00:46.944499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944542    4512 cpu_manager.go:414] "containerMap: removing container" podUID="6452875f-0203-4b2e-8593-7e47cf9710d8" containerName="util"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944546    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6452875f-0203-4b2e-8593-7e47cf9710d8" containerName="util"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944551    4512 cpu_manager.go:414] "containerMap: removing container" podUID="a4405cc0-326d-4057-b87b-0b7c5efc10ae" containerName="util"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944555    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4405cc0-326d-4057-b87b-0b7c5efc10ae" containerName="util"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944560    4512 cpu_manager.go:414] "containerMap: removing container" podUID="54cc3fcf-b55e-49f6-925b-3d9b62b94009" containerName="pull"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944563    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54cc3fcf-b55e-49f6-925b-3d9b62b94009" containerName="pull"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944569    4512 cpu_manager.go:414] "containerMap: removing container" podUID="885425cc-20c2-4f74-b832-a5fe841098d0" containerName="pull"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944573    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="885425cc-20c2-4f74-b832-a5fe841098d0" containerName="pull"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944578    4512 cpu_manager.go:414] "containerMap: removing container" podUID="a6e91bb3-552f-4d57-ad3d-41c1b458e09f" containerName="pull"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944581    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6e91bb3-552f-4d57-ad3d-41c1b458e09f" containerName="pull"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944588    4512 cpu_manager.go:414] "containerMap: removing container" podUID="c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" containerName="pull"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944591    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" containerName="pull"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944638    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="2461de17-c237-444b-9d68-a36b63e2889e" containerName="extract"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944647    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="885425cc-20c2-4f74-b832-a5fe841098d0" containerName="extract"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944653    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="54cc3fcf-b55e-49f6-925b-3d9b62b94009" containerName="extract"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944658    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a0d7d21-bc96-46eb-8370-bdb0cd57620b" containerName="extract"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944663    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4405cc0-326d-4057-b87b-0b7c5efc10ae" containerName="extract"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944668    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0d2cdf6-91b5-43e9-87e1-2c4d711bcc2b" containerName="extract"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944674    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef82cf23-0867-4c91-80d1-11fe242707a8" containerName="extract"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944678    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5de9d8b-4ca3-488a-a727-54f0204173d2" containerName="extract"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944683    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="6452875f-0203-4b2e-8593-7e47cf9710d8" containerName="extract"
Nov 04 02:00:46.945634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.944687    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6e91bb3-552f-4d57-ad3d-41c1b458e09f" containerName="extract"
Nov 04 02:00:46.947024 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.947010    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.000007 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:46.999985    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 02:00:47.000107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.000013    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 02:00:47.000107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.000026    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 02:00:47.000107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.000071    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 02:00:47.000268 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.000254    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 02:00:47.002536 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.000705    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-2szzx\""
Nov 04 02:00:47.051900 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.051869    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61cd7855-400c-49b7-925b-efad3441bb4f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.051982 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.051910    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k8qg\" (UniqueName: \"kubernetes.io/projected/61cd7855-400c-49b7-925b-efad3441bb4f-kube-api-access-5k8qg\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.051982 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.051937    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61cd7855-400c-49b7-925b-efad3441bb4f-manifests\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.052055 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.051988    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61cd7855-400c-49b7-925b-efad3441bb4f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.052138 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.052116    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61cd7855-400c-49b7-925b-efad3441bb4f-tmp\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.052184 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.052163    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/61cd7855-400c-49b7-925b-efad3441bb4f-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.151815 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.151765    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:47.152601 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.152582    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61cd7855-400c-49b7-925b-efad3441bb4f-manifests\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.152644 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.152611    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61cd7855-400c-49b7-925b-efad3441bb4f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.152644 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.152630    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61cd7855-400c-49b7-925b-efad3441bb4f-tmp\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.152754 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.152736    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/61cd7855-400c-49b7-925b-efad3441bb4f-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.152878 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.152859    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61cd7855-400c-49b7-925b-efad3441bb4f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.152935 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.152915    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k8qg\" (UniqueName: \"kubernetes.io/projected/61cd7855-400c-49b7-925b-efad3441bb4f-kube-api-access-5k8qg\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.152992 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.152931    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61cd7855-400c-49b7-925b-efad3441bb4f-manifests\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.155080 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.155060    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61cd7855-400c-49b7-925b-efad3441bb4f-tmp\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.155300 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.155276    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/61cd7855-400c-49b7-925b-efad3441bb4f-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.155590 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.155345    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61cd7855-400c-49b7-925b-efad3441bb4f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.156495 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.156475    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61cd7855-400c-49b7-925b-efad3441bb4f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.180648 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.180619    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq" event={"ID":"a4405cc0-326d-4057-b87b-0b7c5efc10ae","Type":"ContainerDied","Data":"51d6e86584a4178cb7ffed2d98062eb0ce08046028dfa7f12d87f5fce241c1df"}
Nov 04 02:00:47.180734 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.180649    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6hrqq"
Nov 04 02:00:47.180734 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.180686    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51d6e86584a4178cb7ffed2d98062eb0ce08046028dfa7f12d87f5fce241c1df"
Nov 04 02:00:47.245305 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.245285    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"]
Nov 04 02:00:47.260858 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.260838    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k8qg\" (UniqueName: \"kubernetes.io/projected/61cd7855-400c-49b7-925b-efad3441bb4f-kube-api-access-5k8qg\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-x75p2\" (UID: \"61cd7855-400c-49b7-925b-efad3441bb4f\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.385224 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.385197    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:47.560736 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.560710    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:47.598998 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:47.598963    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:48.102226 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:48.102199    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"]
Nov 04 02:00:48.103413 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:48.103383    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61cd7855_400c_49b7_925b_efad3441bb4f.slice/crio-a31b8e4d300dd759711a2a5cf77fb6cd2c246ba5f94facf8399fba2c1aee91b1 WatchSource:0}: Error finding container a31b8e4d300dd759711a2a5cf77fb6cd2c246ba5f94facf8399fba2c1aee91b1: Status 404 returned error can't find the container with id a31b8e4d300dd759711a2a5cf77fb6cd2c246ba5f94facf8399fba2c1aee91b1
Nov 04 02:00:48.105243 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:48.105224    4512 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:00:48.187143 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:48.187113    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2" event={"ID":"61cd7855-400c-49b7-925b-efad3441bb4f","Type":"ContainerStarted","Data":"a31b8e4d300dd759711a2a5cf77fb6cd2c246ba5f94facf8399fba2c1aee91b1"}
Nov 04 02:00:48.392972 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:48.392898    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:48.605960 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:48.605931    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:50.020566 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:50.020458    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:50.284640 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:50.284551    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:50.577181 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:50.577070    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:53.215455 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:53.215375    4512 generic.go:356] "Generic (PLEG): container finished" podID="61cd7855-400c-49b7-925b-efad3441bb4f" containerID="3a3e575519db5fa791edcbee66fb5d06222fe65a2f87e0d0fdb31a9d47fad107" exitCode=0
Nov 04 02:00:53.215838 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:53.215457    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2" event={"ID":"61cd7855-400c-49b7-925b-efad3441bb4f","Type":"ContainerDied","Data":"3a3e575519db5fa791edcbee66fb5d06222fe65a2f87e0d0fdb31a9d47fad107"}
Nov 04 02:00:53.553550 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:53.553492    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:54.221719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.221685    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2" event={"ID":"61cd7855-400c-49b7-925b-efad3441bb4f","Type":"ContainerStarted","Data":"0e1413a831927814b769bfa3856249d78f0b8ed6643426a1e2ec9001f3c2393f"}
Nov 04 02:00:54.222036 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.221873    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:00:54.344397 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.344327    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"]
Nov 04 02:00:54.347245 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.347224    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"
Nov 04 02:00:54.392809 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.392758    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2" podStartSLOduration=2.872534305 podStartE2EDuration="8.392742491s" totalImagesPullingTime="5.0096211s" totalInitContainerRuntime="510.587086ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:54.370841624 +0000 UTC m=+316.176298255" watchObservedRunningTime="2026-11-04 02:00:54.392742491 +0000 UTC m=+316.198199123"
Nov 04 02:00:54.400932 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.400909    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:00:54.401152 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.401135    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Nov 04 02:00:54.401286 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.401273    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-drfhr\""
Nov 04 02:00:54.413597 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.413576    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdggz\" (UniqueName: \"kubernetes.io/projected/bcf51876-a74c-4d0b-8b27-e6204b838b44-kube-api-access-kdggz\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn\" (UID: \"bcf51876-a74c-4d0b-8b27-e6204b838b44\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"
Nov 04 02:00:54.413822 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.413803    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcf51876-a74c-4d0b-8b27-e6204b838b44-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn\" (UID: \"bcf51876-a74c-4d0b-8b27-e6204b838b44\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"
Nov 04 02:00:54.421917 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.421894    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"]
Nov 04 02:00:54.515118 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.515081    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdggz\" (UniqueName: \"kubernetes.io/projected/bcf51876-a74c-4d0b-8b27-e6204b838b44-kube-api-access-kdggz\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn\" (UID: \"bcf51876-a74c-4d0b-8b27-e6204b838b44\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"
Nov 04 02:00:54.515229 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.515135    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcf51876-a74c-4d0b-8b27-e6204b838b44-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn\" (UID: \"bcf51876-a74c-4d0b-8b27-e6204b838b44\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"
Nov 04 02:00:54.515551 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.515498    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcf51876-a74c-4d0b-8b27-e6204b838b44-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn\" (UID: \"bcf51876-a74c-4d0b-8b27-e6204b838b44\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"
Nov 04 02:00:54.576138 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.576116    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdggz\" (UniqueName: \"kubernetes.io/projected/bcf51876-a74c-4d0b-8b27-e6204b838b44-kube-api-access-kdggz\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn\" (UID: \"bcf51876-a74c-4d0b-8b27-e6204b838b44\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"
Nov 04 02:00:54.658780 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:54.658727    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"
Nov 04 02:00:55.087477 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:55.087453    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn"]
Nov 04 02:00:55.089354 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:00:55.089330    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf51876_a74c_4d0b_8b27_e6204b838b44.slice/crio-e8937a99656bd00b8d4cdd383422aa74f4d57d7d196efd04c2c42bbd159e8d03 WatchSource:0}: Error finding container e8937a99656bd00b8d4cdd383422aa74f4d57d7d196efd04c2c42bbd159e8d03: Status 404 returned error can't find the container with id e8937a99656bd00b8d4cdd383422aa74f4d57d7d196efd04c2c42bbd159e8d03
Nov 04 02:00:55.227548 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:55.227494    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn" event={"ID":"bcf51876-a74c-4d0b-8b27-e6204b838b44","Type":"ContainerStarted","Data":"e8937a99656bd00b8d4cdd383422aa74f4d57d7d196efd04c2c42bbd159e8d03"}
Nov 04 02:00:55.422055 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:55.421965    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:55.635810 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:55.635769    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:56.777900 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:00:56.777859    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:00:56.789649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:56.789621    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:57.046855 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:57.046766    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:58.430395 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:58.430349    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:58.744585 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:58.744540    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:59.250815 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:59.250779    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn" event={"ID":"bcf51876-a74c-4d0b-8b27-e6204b838b44","Type":"ContainerStarted","Data":"c5c74589c3f0d044e1fbf4f673126ed1f6fcf32bab8b7b13dd208aa81ac7697b"}
Nov 04 02:00:59.471616 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:59.471561    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-vtxbn" podStartSLOduration=2.09800275 podStartE2EDuration="5.471545789s" totalImagesPullingTime="3.373543039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:59.420739011 +0000 UTC m=+321.226195668" watchObservedRunningTime="2026-11-04 02:00:59.471545789 +0000 UTC m=+321.277002420"
Nov 04 02:00:59.855669 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:59.855633    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"]
Nov 04 02:00:59.859970 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:59.859948    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"
Nov 04 02:00:59.918193 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:59.918164    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 04 02:00:59.918292 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:59.918221    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:00:59.918292 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:59.918255    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-z8x44\""
Nov 04 02:00:59.931704 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:59.931680    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"]
Nov 04 02:00:59.962739 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:59.962711    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a0831f2-2e0a-458a-a499-e3d4fc79ee85-tmp\") pod \"jobset-operator-674cd59dc5-kx5jb\" (UID: \"6a0831f2-2e0a-458a-a499-e3d4fc79ee85\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"
Nov 04 02:00:59.962961 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:00:59.962939    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjn66\" (UniqueName: \"kubernetes.io/projected/6a0831f2-2e0a-458a-a499-e3d4fc79ee85-kube-api-access-vjn66\") pod \"jobset-operator-674cd59dc5-kx5jb\" (UID: \"6a0831f2-2e0a-458a-a499-e3d4fc79ee85\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"
Nov 04 02:01:00.064408 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:00.064373    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjn66\" (UniqueName: \"kubernetes.io/projected/6a0831f2-2e0a-458a-a499-e3d4fc79ee85-kube-api-access-vjn66\") pod \"jobset-operator-674cd59dc5-kx5jb\" (UID: \"6a0831f2-2e0a-458a-a499-e3d4fc79ee85\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"
Nov 04 02:01:00.064622 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:00.064480    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a0831f2-2e0a-458a-a499-e3d4fc79ee85-tmp\") pod \"jobset-operator-674cd59dc5-kx5jb\" (UID: \"6a0831f2-2e0a-458a-a499-e3d4fc79ee85\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"
Nov 04 02:01:00.064897 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:00.064874    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a0831f2-2e0a-458a-a499-e3d4fc79ee85-tmp\") pod \"jobset-operator-674cd59dc5-kx5jb\" (UID: \"6a0831f2-2e0a-458a-a499-e3d4fc79ee85\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"
Nov 04 02:01:00.150870 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:00.150792    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjn66\" (UniqueName: \"kubernetes.io/projected/6a0831f2-2e0a-458a-a499-e3d4fc79ee85-kube-api-access-vjn66\") pod \"jobset-operator-674cd59dc5-kx5jb\" (UID: \"6a0831f2-2e0a-458a-a499-e3d4fc79ee85\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"
Nov 04 02:01:00.179535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:00.179487    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"
Nov 04 02:01:00.289163 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:00.289119    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:00.563308 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:00.563281    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb"]
Nov 04 02:01:00.564478 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:01:00.564451    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a0831f2_2e0a_458a_a499_e3d4fc79ee85.slice/crio-7adae26cfe86b53e6559415e68e7dcbfaf45eaa7555ba7d750c3751102e76952 WatchSource:0}: Error finding container 7adae26cfe86b53e6559415e68e7dcbfaf45eaa7555ba7d750c3751102e76952: Status 404 returned error can't find the container with id 7adae26cfe86b53e6559415e68e7dcbfaf45eaa7555ba7d750c3751102e76952
Nov 04 02:01:00.632704 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:00.632667    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:01.263033 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:01.262997    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb" event={"ID":"6a0831f2-2e0a-458a-a499-e3d4fc79ee85","Type":"ContainerStarted","Data":"7adae26cfe86b53e6559415e68e7dcbfaf45eaa7555ba7d750c3751102e76952"}
Nov 04 02:01:02.625880 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:02.625835    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:03.103964 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:03.103928    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:05.230419 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:05.230386    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2"
Nov 04 02:01:05.766035 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:05.766004    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:06.399060 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:06.399014    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:06.594036 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:06.593994    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:07.295135 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:07.295097    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb" event={"ID":"6a0831f2-2e0a-458a-a499-e3d4fc79ee85","Type":"ContainerStarted","Data":"2f41c4c7e595971ca5298ce120e68731d81f80a11d6f11e2ee01dd87c498947d"}
Nov 04 02:01:07.411189 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:07.411132    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb" podStartSLOduration=2.333001131 podStartE2EDuration="8.411116479s" totalImagesPullingTime="6.078115348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:07.409761656 +0000 UTC m=+329.215218310" watchObservedRunningTime="2026-11-04 02:01:07.411116479 +0000 UTC m=+329.216573111"
Nov 04 02:01:09.349048 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.349009    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j"]
Nov 04 02:01:09.351454 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.351438    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j"
Nov 04 02:01:09.405188 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.405161    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-j7z8d\""
Nov 04 02:01:09.405361 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.405190    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 02:01:09.449564 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.449525    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtm9v\" (UniqueName: \"kubernetes.io/projected/3802223a-2462-41a3-98ef-884d9dcc36f2-kube-api-access-vtm9v\") pod \"limitador-operator-controller-manager-5778bb4449-htz9j\" (UID: \"3802223a-2462-41a3-98ef-884d9dcc36f2\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j"
Nov 04 02:01:09.454878 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.454855    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 02:01:09.456940 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.456919    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j"]
Nov 04 02:01:09.550721 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.550681    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtm9v\" (UniqueName: \"kubernetes.io/projected/3802223a-2462-41a3-98ef-884d9dcc36f2-kube-api-access-vtm9v\") pod \"limitador-operator-controller-manager-5778bb4449-htz9j\" (UID: \"3802223a-2462-41a3-98ef-884d9dcc36f2\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j"
Nov 04 02:01:09.619333 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.619258    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtm9v\" (UniqueName: \"kubernetes.io/projected/3802223a-2462-41a3-98ef-884d9dcc36f2-kube-api-access-vtm9v\") pod \"limitador-operator-controller-manager-5778bb4449-htz9j\" (UID: \"3802223a-2462-41a3-98ef-884d9dcc36f2\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j"
Nov 04 02:01:09.664164 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.664126    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j"
Nov 04 02:01:09.797353 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:09.797323    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:10.005302 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:10.005238    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:10.045916 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:10.045880    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j"]
Nov 04 02:01:10.075603 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:01:10.075573    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3802223a_2462_41a3_98ef_884d9dcc36f2.slice/crio-0f0637ec35ae48727e78a0f9fd5f98f04c3b141b2cb3f89b1172635ed13957de WatchSource:0}: Error finding container 0f0637ec35ae48727e78a0f9fd5f98f04c3b141b2cb3f89b1172635ed13957de: Status 404 returned error can't find the container with id 0f0637ec35ae48727e78a0f9fd5f98f04c3b141b2cb3f89b1172635ed13957de
Nov 04 02:01:10.313357 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:10.313275    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j" event={"ID":"3802223a-2462-41a3-98ef-884d9dcc36f2","Type":"ContainerStarted","Data":"0f0637ec35ae48727e78a0f9fd5f98f04c3b141b2cb3f89b1172635ed13957de"}
Nov 04 02:01:10.580588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:10.579257    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:10.954352 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:10.954273    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-8dffj"]
Nov 04 02:01:10.957451 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:10.957432    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-8dffj"
Nov 04 02:01:11.052143 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.052111    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:11.059038 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.059016    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-697ng\""
Nov 04 02:01:11.059038 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.059025    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 04 02:01:11.059183 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.059023    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 04 02:01:11.061082 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.061062    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-8dffj"]
Nov 04 02:01:11.064437 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.064411    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df374159-7e12-431d-9f8d-864d8a95e730-bound-sa-token\") pod \"cert-manager-597b95cb84-8dffj\" (UID: \"df374159-7e12-431d-9f8d-864d8a95e730\") " pod="cert-manager/cert-manager-597b95cb84-8dffj"
Nov 04 02:01:11.064569 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.064441    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2xkb\" (UniqueName: \"kubernetes.io/projected/df374159-7e12-431d-9f8d-864d8a95e730-kube-api-access-p2xkb\") pod \"cert-manager-597b95cb84-8dffj\" (UID: \"df374159-7e12-431d-9f8d-864d8a95e730\") " pod="cert-manager/cert-manager-597b95cb84-8dffj"
Nov 04 02:01:11.165484 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.165446    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df374159-7e12-431d-9f8d-864d8a95e730-bound-sa-token\") pod \"cert-manager-597b95cb84-8dffj\" (UID: \"df374159-7e12-431d-9f8d-864d8a95e730\") " pod="cert-manager/cert-manager-597b95cb84-8dffj"
Nov 04 02:01:11.165484 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.165483    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2xkb\" (UniqueName: \"kubernetes.io/projected/df374159-7e12-431d-9f8d-864d8a95e730-kube-api-access-p2xkb\") pod \"cert-manager-597b95cb84-8dffj\" (UID: \"df374159-7e12-431d-9f8d-864d8a95e730\") " pod="cert-manager/cert-manager-597b95cb84-8dffj"
Nov 04 02:01:11.228941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.228876    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df374159-7e12-431d-9f8d-864d8a95e730-bound-sa-token\") pod \"cert-manager-597b95cb84-8dffj\" (UID: \"df374159-7e12-431d-9f8d-864d8a95e730\") " pod="cert-manager/cert-manager-597b95cb84-8dffj"
Nov 04 02:01:11.229180 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.229155    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2xkb\" (UniqueName: \"kubernetes.io/projected/df374159-7e12-431d-9f8d-864d8a95e730-kube-api-access-p2xkb\") pod \"cert-manager-597b95cb84-8dffj\" (UID: \"df374159-7e12-431d-9f8d-864d8a95e730\") " pod="cert-manager/cert-manager-597b95cb84-8dffj"
Nov 04 02:01:11.269282 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.269251    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-8dffj"
Nov 04 02:01:11.639175 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:11.639146    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-8dffj"]
Nov 04 02:01:11.640882 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:01:11.640852    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf374159_7e12_431d_9f8d_864d8a95e730.slice/crio-2e3baa1cb5099f650f015159ce476284abe16bac677800b7bbef28482f464efb WatchSource:0}: Error finding container 2e3baa1cb5099f650f015159ce476284abe16bac677800b7bbef28482f464efb: Status 404 returned error can't find the container with id 2e3baa1cb5099f650f015159ce476284abe16bac677800b7bbef28482f464efb
Nov 04 02:01:12.326682 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:12.326626    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-8dffj" event={"ID":"df374159-7e12-431d-9f8d-864d8a95e730","Type":"ContainerStarted","Data":"2e3baa1cb5099f650f015159ce476284abe16bac677800b7bbef28482f464efb"}
Nov 04 02:01:12.612996 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:12.612916    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:14.026539 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:14.026488    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-tx25t"]
Nov 04 02:01:14.032566 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:14.032540    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-tx25t"
Nov 04 02:01:14.139805 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:14.139768    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-wkbc2\""
Nov 04 02:01:14.142492 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:14.142462    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-tx25t"]
Nov 04 02:01:14.194292 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:14.194262    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsgnp\" (UniqueName: \"kubernetes.io/projected/848ca595-eb11-49f4-bf0d-77d303fb0432-kube-api-access-vsgnp\") pod \"authorino-operator-7b7bb4bc77-tx25t\" (UID: \"848ca595-eb11-49f4-bf0d-77d303fb0432\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-tx25t"
Nov 04 02:01:14.295407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:14.295323    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsgnp\" (UniqueName: \"kubernetes.io/projected/848ca595-eb11-49f4-bf0d-77d303fb0432-kube-api-access-vsgnp\") pod \"authorino-operator-7b7bb4bc77-tx25t\" (UID: \"848ca595-eb11-49f4-bf0d-77d303fb0432\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-tx25t"
Nov 04 02:01:14.357094 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:14.357057    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsgnp\" (UniqueName: \"kubernetes.io/projected/848ca595-eb11-49f4-bf0d-77d303fb0432-kube-api-access-vsgnp\") pod \"authorino-operator-7b7bb4bc77-tx25t\" (UID: \"848ca595-eb11-49f4-bf0d-77d303fb0432\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-tx25t"
Nov 04 02:01:14.648200 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:14.648145    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-tx25t"
Nov 04 02:01:14.779824 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:14.779793    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:15.031360 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:15.031326    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-tx25t"]
Nov 04 02:01:15.067257 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:01:15.067226    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod848ca595_eb11_49f4_bf0d_77d303fb0432.slice/crio-691d3a801546af131e715d64391efa5034b527dd0065d8376bac8b6fbbe7e967 WatchSource:0}: Error finding container 691d3a801546af131e715d64391efa5034b527dd0065d8376bac8b6fbbe7e967: Status 404 returned error can't find the container with id 691d3a801546af131e715d64391efa5034b527dd0065d8376bac8b6fbbe7e967
Nov 04 02:01:15.346358 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:15.346266    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-tx25t" event={"ID":"848ca595-eb11-49f4-bf0d-77d303fb0432","Type":"ContainerStarted","Data":"691d3a801546af131e715d64391efa5034b527dd0065d8376bac8b6fbbe7e967"}
Nov 04 02:01:15.348324 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:15.348297    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j" event={"ID":"3802223a-2462-41a3-98ef-884d9dcc36f2","Type":"ContainerStarted","Data":"23bd443547135d2660cd8b83f2f84e87bfda20b80b7d7c2f6be9674a4a24bbf1"}
Nov 04 02:01:15.348464 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:15.348378    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j"
Nov 04 02:01:15.381005 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:15.380963    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:15.468604 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:15.468551    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j" podStartSLOduration=1.919316633 podStartE2EDuration="6.468537765s" totalImagesPullingTime="4.549221132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:15.465813909 +0000 UTC m=+337.271270542" watchObservedRunningTime="2026-11-04 02:01:15.468537765 +0000 UTC m=+337.273994393"
Nov 04 02:01:15.976446 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:15.976405    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:16.332260 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:16.332229    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:16.355064 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:16.355030    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-8dffj" event={"ID":"df374159-7e12-431d-9f8d-864d8a95e730","Type":"ContainerStarted","Data":"13d2bb54b03326d998a1a8a9633c560e573cc3773e22aa18282f8e35f0c57c44"}
Nov 04 02:01:16.471916 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:16.471856    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-8dffj" podStartSLOduration=1.931655176 podStartE2EDuration="6.471841544s" totalImagesPullingTime="4.540186368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:16.469564677 +0000 UTC m=+338.275021309" watchObservedRunningTime="2026-11-04 02:01:16.471841544 +0000 UTC m=+338.277298175"
Nov 04 02:01:17.465403 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:17.465361    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:17.782204 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:17.782165    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:19.370414 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:19.370374    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-tx25t" event={"ID":"848ca595-eb11-49f4-bf0d-77d303fb0432","Type":"ContainerStarted","Data":"c7c0dfd978c71d5911c2c770c34e0a5477440f8e0b063e668586ba92ea2b359d"}
Nov 04 02:01:19.370874 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:19.370525    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-tx25t"
Nov 04 02:01:19.487316 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:19.487259    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-tx25t" podStartSLOduration=3.18353309 podStartE2EDuration="6.487245584s" totalImagesPullingTime="3.303712494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:19.485982935 +0000 UTC m=+341.291439570" watchObservedRunningTime="2026-11-04 02:01:19.487245584 +0000 UTC m=+341.292702216"
Nov 04 02:01:20.186991 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.186957    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"]
Nov 04 02:01:20.189337 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.189321    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"
Nov 04 02:01:20.249373 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.249334    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbwcl\" (UniqueName: \"kubernetes.io/projected/c557873e-e31e-4447-8ba1-e8a84c3cc5f6-kube-api-access-hbwcl\") pod \"kuadrant-operator-controller-manager-678d87766f-g65d7\" (UID: \"c557873e-e31e-4447-8ba1-e8a84c3cc5f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"
Nov 04 02:01:20.249373 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.249371    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c557873e-e31e-4447-8ba1-e8a84c3cc5f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-g65d7\" (UID: \"c557873e-e31e-4447-8ba1-e8a84c3cc5f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"
Nov 04 02:01:20.291991 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.291956    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2nrt8\""
Nov 04 02:01:20.294189 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.294162    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"]
Nov 04 02:01:20.350015 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.349976    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbwcl\" (UniqueName: \"kubernetes.io/projected/c557873e-e31e-4447-8ba1-e8a84c3cc5f6-kube-api-access-hbwcl\") pod \"kuadrant-operator-controller-manager-678d87766f-g65d7\" (UID: \"c557873e-e31e-4447-8ba1-e8a84c3cc5f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"
Nov 04 02:01:20.350015 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.350018    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c557873e-e31e-4447-8ba1-e8a84c3cc5f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-g65d7\" (UID: \"c557873e-e31e-4447-8ba1-e8a84c3cc5f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"
Nov 04 02:01:20.350475 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.350449    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c557873e-e31e-4447-8ba1-e8a84c3cc5f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-g65d7\" (UID: \"c557873e-e31e-4447-8ba1-e8a84c3cc5f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"
Nov 04 02:01:20.418524 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.418474    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbwcl\" (UniqueName: \"kubernetes.io/projected/c557873e-e31e-4447-8ba1-e8a84c3cc5f6-kube-api-access-hbwcl\") pod \"kuadrant-operator-controller-manager-678d87766f-g65d7\" (UID: \"c557873e-e31e-4447-8ba1-e8a84c3cc5f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"
Nov 04 02:01:20.500484 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.500434    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"
Nov 04 02:01:20.928874 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:20.928848    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"]
Nov 04 02:01:20.930981 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:01:20.930948    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc557873e_e31e_4447_8ba1_e8a84c3cc5f6.slice/crio-b8ab55494176c644fdceabf9e80afce890e9718e91bb7a0030c896bc34b45c9e WatchSource:0}: Error finding container b8ab55494176c644fdceabf9e80afce890e9718e91bb7a0030c896bc34b45c9e: Status 404 returned error can't find the container with id b8ab55494176c644fdceabf9e80afce890e9718e91bb7a0030c896bc34b45c9e
Nov 04 02:01:21.178405 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:21.178316    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:21.381400 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:21.381366    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7" event={"ID":"c557873e-e31e-4447-8ba1-e8a84c3cc5f6","Type":"ContainerStarted","Data":"b8ab55494176c644fdceabf9e80afce890e9718e91bb7a0030c896bc34b45c9e"}
Nov 04 02:01:23.252648 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:23.252609    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:24.793716 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:24.793677    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:25.055118 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:25.055042    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:25.705223 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:25.705178    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:25.965780 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:25.965688    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:26.357743 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:26.357710    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j"
Nov 04 02:01:26.778937 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:01:26.778900    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:01:28.782804 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:28.782768    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:29.040674 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:29.040588    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:29.244952 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:29.244919    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:30.378930 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:30.378894    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-tx25t"
Nov 04 02:01:31.441585 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:31.441552    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7" event={"ID":"c557873e-e31e-4447-8ba1-e8a84c3cc5f6","Type":"ContainerStarted","Data":"c220d56f0f5d03779eb1c43bc776ac69aa43a8cab0e726a4230fbc7987372016"}
Nov 04 02:01:31.442062 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:31.441670    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"
Nov 04 02:01:33.647847 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:33.647815    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:33.851240 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:33.851201    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:35.245840 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:35.245785    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:35.455167 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:35.455128    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:35.993824 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:35.993785    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:36.980244 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:36.980210    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:37.343016 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:37.342980    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:37.665179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:37.664999    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:37.916104 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:37.916009    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:38.090084 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.090012    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7" podStartSLOduration=8.307763116 podStartE2EDuration="18.089992111s" totalImagesPullingTime="9.782228995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:31.672179102 +0000 UTC m=+353.477635734" watchObservedRunningTime="2026-11-04 02:01:38.089992111 +0000 UTC m=+359.895448744"
Nov 04 02:01:38.092113 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.092089    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"]
Nov 04 02:01:38.095861 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.095820    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.130921 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.130870    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:38.148591 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.148566    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-c484d\""
Nov 04 02:01:38.199914 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.199836    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"]
Nov 04 02:01:38.224527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.224475    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.224678 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.224546    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.224678 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.224648    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.224773 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.224688    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.224773 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.224720    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.224773 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.224767    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.224870 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.224848    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.224903 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.224872    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.224903 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.224893    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wncps\" (UniqueName: \"kubernetes.io/projected/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-kube-api-access-wncps\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325372 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325339    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325580 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325379    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325580 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325405    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wncps\" (UniqueName: \"kubernetes.io/projected/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-kube-api-access-wncps\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325580 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325465    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325580 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325488    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325580 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325546    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325856 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325577    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325856 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325605    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325856 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325633    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325856 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325821    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.325856 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325821    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.326112 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.325912    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.326112 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.326075    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.326371 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.326349    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.327992 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.327959    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.327992 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.327983    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.346999 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.346965    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:38.351320 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.351296    4512 scope.go:122] "RemoveContainer" containerID="7ea8b6a28f05012a07dab156f123273663730cd0c2119c08ab6c2aa30d5bae6a"
Nov 04 02:01:38.386031 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.386004    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.386174 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.386152    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wncps\" (UniqueName: \"kubernetes.io/projected/9bde987c-e9ca-48e4-b60a-e6b394e3ea6e-kube-api-access-wncps\") pod \"data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk\" (UID: \"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.460459 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.460373    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-c484d\""
Nov 04 02:01:38.469984 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.469967    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:38.551282 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.551248    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-n26k6"]
Nov 04 02:01:38.552719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.552690    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:38.555165 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.555146    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-n26k6"
Nov 04 02:01:38.607933 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.607852    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xxnr6\""
Nov 04 02:01:38.610540 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.610499    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-n26k6"]
Nov 04 02:01:38.628561 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.628500    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqjx4\" (UniqueName: \"kubernetes.io/projected/4a7252b3-b6a1-4e6a-9894-ff640bcf128c-kube-api-access-cqjx4\") pod \"authorino-57f7f8659f-n26k6\" (UID: \"4a7252b3-b6a1-4e6a-9894-ff640bcf128c\") " pod="kuadrant-system/authorino-57f7f8659f-n26k6"
Nov 04 02:01:38.729744 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.729662    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqjx4\" (UniqueName: \"kubernetes.io/projected/4a7252b3-b6a1-4e6a-9894-ff640bcf128c-kube-api-access-cqjx4\") pod \"authorino-57f7f8659f-n26k6\" (UID: \"4a7252b3-b6a1-4e6a-9894-ff640bcf128c\") " pod="kuadrant-system/authorino-57f7f8659f-n26k6"
Nov 04 02:01:38.763203 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.763171    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:38.792802 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.792777    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqjx4\" (UniqueName: \"kubernetes.io/projected/4a7252b3-b6a1-4e6a-9894-ff640bcf128c-kube-api-access-cqjx4\") pod \"authorino-57f7f8659f-n26k6\" (UID: \"4a7252b3-b6a1-4e6a-9894-ff640bcf128c\") " pod="kuadrant-system/authorino-57f7f8659f-n26k6"
Nov 04 02:01:38.869918 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.869889    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-n26k6"
Nov 04 02:01:38.902016 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.901983    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"]
Nov 04 02:01:38.905571 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:01:38.905544    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bde987c_e9ca_48e4_b60a_e6b394e3ea6e.slice/crio-71203fc13e6327323814c9c79aedcb702b45a1965673c0651be440a7ef90070b WatchSource:0}: Error finding container 71203fc13e6327323814c9c79aedcb702b45a1965673c0651be440a7ef90070b: Status 404 returned error can't find the container with id 71203fc13e6327323814c9c79aedcb702b45a1965673c0651be440a7ef90070b
Nov 04 02:01:38.907876 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.907843    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:01:38.907965 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.907907    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:01:38.907965 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.907937    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:01:38.972790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:38.972753    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:39.245521 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:39.245484    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-n26k6"]
Nov 04 02:01:39.247334 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:01:39.247309    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a7252b3_b6a1_4e6a_9894_ff640bcf128c.slice/crio-8921a1f36b628f63a89fbd712175d34cacfe7e698262f8f079acef1c84c19800 WatchSource:0}: Error finding container 8921a1f36b628f63a89fbd712175d34cacfe7e698262f8f079acef1c84c19800: Status 404 returned error can't find the container with id 8921a1f36b628f63a89fbd712175d34cacfe7e698262f8f079acef1c84c19800
Nov 04 02:01:39.487050 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:39.487010    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk" event={"ID":"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e","Type":"ContainerStarted","Data":"194fb099212cecb899db16746b9b2e9b91ed25235e53587ef5dfee2407f0599c"}
Nov 04 02:01:39.487243 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:39.487071    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk" event={"ID":"9bde987c-e9ca-48e4-b60a-e6b394e3ea6e","Type":"ContainerStarted","Data":"71203fc13e6327323814c9c79aedcb702b45a1965673c0651be440a7ef90070b"}
Nov 04 02:01:39.488207 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:39.488170    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-n26k6" event={"ID":"4a7252b3-b6a1-4e6a-9894-ff640bcf128c","Type":"ContainerStarted","Data":"8921a1f36b628f63a89fbd712175d34cacfe7e698262f8f079acef1c84c19800"}
Nov 04 02:01:39.607281 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:39.607165    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk" podStartSLOduration=1.607150919 podStartE2EDuration="1.607150919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:39.606047911 +0000 UTC m=+361.411504545" watchObservedRunningTime="2026-11-04 02:01:39.607150919 +0000 UTC m=+361.412607550"
Nov 04 02:01:40.227336 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:40.227296    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:40.470162 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:40.470134    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:40.470674 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:40.470169    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:40.475011 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:40.474988    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:40.494206 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:40.494178    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:40.494956 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:40.494938    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk"
Nov 04 02:01:42.449238 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:42.449207    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7"
Nov 04 02:01:43.512028 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:43.511990    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-n26k6" event={"ID":"4a7252b3-b6a1-4e6a-9894-ff640bcf128c","Type":"ContainerStarted","Data":"4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c"}
Nov 04 02:01:43.637218 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:43.636957    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-n26k6" podStartSLOduration=1.874668802 podStartE2EDuration="5.636938549s" totalImagesPullingTime="3.762269747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:43.631203692 +0000 UTC m=+365.436660323" watchObservedRunningTime="2026-11-04 02:01:43.636938549 +0000 UTC m=+365.442395185"
Nov 04 02:01:44.587936 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:44.587897    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:44.871114 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:44.871026    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:45.594977 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:45.594947    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:46.541043 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:46.541015    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-n26k6"]
Nov 04 02:01:46.541213 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:46.541181    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-n26k6" podUID="4a7252b3-b6a1-4e6a-9894-ff640bcf128c" containerName="authorino" containerID="cri-o://4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c" gracePeriod=30
Nov 04 02:01:46.820446 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:46.820389    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-n26k6"
Nov 04 02:01:46.916527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:46.916473    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a7252b3-b6a1-4e6a-9894-ff640bcf128c-kube-api-access-cqjx4\" (UniqueName: \"kubernetes.io/projected/4a7252b3-b6a1-4e6a-9894-ff640bcf128c-kube-api-access-cqjx4\") pod \"4a7252b3-b6a1-4e6a-9894-ff640bcf128c\" (UID: \"4a7252b3-b6a1-4e6a-9894-ff640bcf128c\") "
Nov 04 02:01:46.918702 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:46.918679    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a7252b3-b6a1-4e6a-9894-ff640bcf128c-kube-api-access-cqjx4" pod "4a7252b3-b6a1-4e6a-9894-ff640bcf128c" (UID: "4a7252b3-b6a1-4e6a-9894-ff640bcf128c"). InnerVolumeSpecName "kube-api-access-cqjx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:01:47.017492 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:47.017460    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqjx4\" (UniqueName: \"kubernetes.io/projected/4a7252b3-b6a1-4e6a-9894-ff640bcf128c-kube-api-access-cqjx4\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:01:47.534524 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:47.534474    4512 generic.go:356] "Generic (PLEG): container finished" podID="4a7252b3-b6a1-4e6a-9894-ff640bcf128c" containerID="4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c" exitCode=0
Nov 04 02:01:47.534722 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:47.534528    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-n26k6"
Nov 04 02:01:47.534722 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:47.534553    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-n26k6" event={"ID":"4a7252b3-b6a1-4e6a-9894-ff640bcf128c","Type":"ContainerDied","Data":"4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c"}
Nov 04 02:01:47.534722 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:47.534599    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-n26k6" event={"ID":"4a7252b3-b6a1-4e6a-9894-ff640bcf128c","Type":"ContainerDied","Data":"8921a1f36b628f63a89fbd712175d34cacfe7e698262f8f079acef1c84c19800"}
Nov 04 02:01:47.534722 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:47.534617    4512 scope.go:122] "RemoveContainer" containerID="4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c"
Nov 04 02:01:47.544154 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:47.544018    4512 scope.go:122] "RemoveContainer" containerID="4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c"
Nov 04 02:01:47.544309 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:01:47.544288    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c\": container with ID starting with 4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c not found: ID does not exist" containerID="4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c"
Nov 04 02:01:47.544353 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:47.544319    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c"} err="failed to get container status \"4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c\": rpc error: code = NotFound desc = could not find container \"4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c\": container with ID starting with 4226a805473fbbdbf6447dab11c8ff7bf9e62688b0765d2a1d7d02afde08dd6c not found: ID does not exist"
Nov 04 02:01:47.697349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:47.697323    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-n26k6"]
Nov 04 02:01:47.701235 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:47.701215    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-n26k6"]
Nov 04 02:01:48.288574 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:48.288528    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:48.310893 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:48.310857    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a7252b3-b6a1-4e6a-9894-ff640bcf128c" path="/var/lib/kubelet/pods/4a7252b3-b6a1-4e6a-9894-ff640bcf128c/volumes"
Nov 04 02:01:48.495126 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:48.495088    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:48.702246 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:48.702158    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:50.538391 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:50.538353    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:50.738928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:50.738888    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:53.893485 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:53.893436    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:54.111902 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:54.111869    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:54.320824 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:01:54.320791    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:56.777884 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:01:56.777842    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:02:00.337294 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:00.337262    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:00.584107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:00.584069    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:03.316285 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:03.316247    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:03.565645 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:03.565613    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:04.037498 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.037462    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"]
Nov 04 02:02:04.037878 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.037865    4512 cpu_manager.go:414] "containerMap: removing container" podUID="4a7252b3-b6a1-4e6a-9894-ff640bcf128c" containerName="authorino"
Nov 04 02:02:04.037921 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.037880    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a7252b3-b6a1-4e6a-9894-ff640bcf128c" containerName="authorino"
Nov 04 02:02:04.037959 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.037955    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a7252b3-b6a1-4e6a-9894-ff640bcf128c" containerName="authorino"
Nov 04 02:02:04.040070 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.040050    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"
Nov 04 02:02:04.060328 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.060285    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5swzv\" (UniqueName: \"kubernetes.io/projected/4805db53-c126-426f-b0e1-9e051e33c509-kube-api-access-5swzv\") pod \"postgres-7cdbf6dc44-s2h6f\" (UID: \"4805db53-c126-426f-b0e1-9e051e33c509\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"
Nov 04 02:02:04.060471 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.060418    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4805db53-c126-426f-b0e1-9e051e33c509-data\") pod \"postgres-7cdbf6dc44-s2h6f\" (UID: \"4805db53-c126-426f-b0e1-9e051e33c509\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"
Nov 04 02:02:04.092372 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.092343    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 02:02:04.092372 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.092363    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 02:02:04.143933 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.143896    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"]
Nov 04 02:02:04.156035 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.156007    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Nov 04 02:02:04.161416 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.161382    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4805db53-c126-426f-b0e1-9e051e33c509-data\") pod \"postgres-7cdbf6dc44-s2h6f\" (UID: \"4805db53-c126-426f-b0e1-9e051e33c509\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"
Nov 04 02:02:04.161577 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.161434    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5swzv\" (UniqueName: \"kubernetes.io/projected/4805db53-c126-426f-b0e1-9e051e33c509-kube-api-access-5swzv\") pod \"postgres-7cdbf6dc44-s2h6f\" (UID: \"4805db53-c126-426f-b0e1-9e051e33c509\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"
Nov 04 02:02:04.161883 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.161864    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4805db53-c126-426f-b0e1-9e051e33c509-data\") pod \"postgres-7cdbf6dc44-s2h6f\" (UID: \"4805db53-c126-426f-b0e1-9e051e33c509\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"
Nov 04 02:02:04.185424 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.185393    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-6nrvq\""
Nov 04 02:02:04.191299 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.191182    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:04.222222 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.222194    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5swzv\" (UniqueName: \"kubernetes.io/projected/4805db53-c126-426f-b0e1-9e051e33c509-kube-api-access-5swzv\") pod \"postgres-7cdbf6dc44-s2h6f\" (UID: \"4805db53-c126-426f-b0e1-9e051e33c509\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"
Nov 04 02:02:04.352738 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.352648    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"
Nov 04 02:02:04.704283 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.704197    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"]
Nov 04 02:02:04.719131 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.719102    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.732426 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.732399    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"]
Nov 04 02:02:04.767182 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.767143    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.767436 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.767198    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.767436 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.767268    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.767560 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.767428    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t969k\" (UniqueName: \"kubernetes.io/projected/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-kube-api-access-t969k\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.771684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.771651    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-m9hwh\""
Nov 04 02:02:04.771814 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.771723    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 02:02:04.771814 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.771786    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 02:02:04.868287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.868244    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t969k\" (UniqueName: \"kubernetes.io/projected/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-kube-api-access-t969k\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.868493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.868313    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.868493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.868354    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.868493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.868372    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.868740 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.868720    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.868782 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.868755    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.871637 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.871617    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Nov 04 02:02:04.874105 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.874079    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"]
Nov 04 02:02:04.880421 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.880398    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:04.937826 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:04.937793    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t969k\" (UniqueName: \"kubernetes.io/projected/35c6a7e4-1ffb-45d1-86ed-9707bf91b699-kube-api-access-t969k\") pod \"kserve-module-controller-manager-7566cf5fc8-jtkx7\" (UID: \"35c6a7e4-1ffb-45d1-86ed-9707bf91b699\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:05.053996 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:05.053954    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:05.491828 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:05.491798    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"]
Nov 04 02:02:05.493277 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:02:05.493242    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c6a7e4_1ffb_45d1_86ed_9707bf91b699.slice/crio-113d0d04567db81bf39184400afe1450ce2cbcc5a89deb9b9f1900b5226a985a WatchSource:0}: Error finding container 113d0d04567db81bf39184400afe1450ce2cbcc5a89deb9b9f1900b5226a985a: Status 404 returned error can't find the container with id 113d0d04567db81bf39184400afe1450ce2cbcc5a89deb9b9f1900b5226a985a
Nov 04 02:02:05.629535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:05.629481    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7" event={"ID":"35c6a7e4-1ffb-45d1-86ed-9707bf91b699","Type":"ContainerStarted","Data":"113d0d04567db81bf39184400afe1450ce2cbcc5a89deb9b9f1900b5226a985a"}
Nov 04 02:02:05.630772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:05.630742    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f" event={"ID":"4805db53-c126-426f-b0e1-9e051e33c509","Type":"ContainerStarted","Data":"d57a254ef712155ad525c1a32a9378e518bc55d3537b13b23604f9f8cb22beba"}
Nov 04 02:02:09.662041 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:09.661950    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7" event={"ID":"35c6a7e4-1ffb-45d1-86ed-9707bf91b699","Type":"ContainerStarted","Data":"0f6efdaab66e7b9f2c3de251380d24523933b0b3eb17e10a0eb06b7454de18d9"}
Nov 04 02:02:09.662563 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:09.662058    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:09.831002 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:09.830939    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7" podStartSLOduration=1.987037589 podStartE2EDuration="5.830922942s" totalImagesPullingTime="3.843885353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:02:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:02:09.82949582 +0000 UTC m=+391.634952454" watchObservedRunningTime="2026-11-04 02:02:09.830922942 +0000 UTC m=+391.636379576"
Nov 04 02:02:11.675032 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:11.674993    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f" event={"ID":"4805db53-c126-426f-b0e1-9e051e33c509","Type":"ContainerStarted","Data":"efed69609f215befab941f737f427e44911dd03b296f0908a4a9ecc0bd87342a"}
Nov 04 02:02:11.675417 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:11.675175    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"
Nov 04 02:02:11.790148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:11.790089    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f" podStartSLOduration=1.9789578479999999 podStartE2EDuration="8.790069129s" totalImagesPullingTime="6.811111281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:02:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:02:11.789641274 +0000 UTC m=+393.595097907" watchObservedRunningTime="2026-11-04 02:02:11.790069129 +0000 UTC m=+393.595525764"
Nov 04 02:02:17.710381 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:17.710353    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-s2h6f"
Nov 04 02:02:26.193654 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:26.193568    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:26.525820 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:26.525785    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:26.777635 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:02:26.777553    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:02:30.670049 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:30.670020    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7"
Nov 04 02:02:35.384150 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.384112    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:35.403131 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.403099    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-594b9656dc-jtm5p"]
Nov 04 02:02:35.406886 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.406863    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:35.459099 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.459067    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 04 02:02:35.459197 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.459179    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 04 02:02:35.462445 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.462418    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 04 02:02:35.462588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.462493    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9zvl8\""
Nov 04 02:02:35.511102 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.511072    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-594b9656dc-jtm5p"]
Nov 04 02:02:35.568353 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.568320    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ktv5\" (UniqueName: \"kubernetes.io/projected/114eee1c-8a1a-4d02-82b2-b2a08af9f389-kube-api-access-7ktv5\") pod \"maas-controller-594b9656dc-jtm5p\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") " pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:35.568558 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.568358    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-metrics-certs\") pod \"maas-controller-594b9656dc-jtm5p\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") " pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:35.568558 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.568400    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-webhook-cert\") pod \"maas-controller-594b9656dc-jtm5p\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") " pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:35.631255 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.631223    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:35.669137 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.669066    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ktv5\" (UniqueName: \"kubernetes.io/projected/114eee1c-8a1a-4d02-82b2-b2a08af9f389-kube-api-access-7ktv5\") pod \"maas-controller-594b9656dc-jtm5p\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") " pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:35.669137 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.669104    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-metrics-certs\") pod \"maas-controller-594b9656dc-jtm5p\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") " pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:35.669286 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.669142    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-webhook-cert\") pod \"maas-controller-594b9656dc-jtm5p\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") " pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:35.671754 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.671725    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-webhook-cert\") pod \"maas-controller-594b9656dc-jtm5p\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") " pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:35.671754 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.671747    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-metrics-certs\") pod \"maas-controller-594b9656dc-jtm5p\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") " pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:35.730426 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.730384    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ktv5\" (UniqueName: \"kubernetes.io/projected/114eee1c-8a1a-4d02-82b2-b2a08af9f389-kube-api-access-7ktv5\") pod \"maas-controller-594b9656dc-jtm5p\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") " pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:35.832634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:35.832599    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:36.020124 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.020092    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:36.031280 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.031252    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-575cb5b89b-xd5wg"]
Nov 04 02:02:36.035181 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.035136    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.096101 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.095766    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-575cb5b89b-xd5wg"]
Nov 04 02:02:36.173042 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.173009    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t92s\" (UniqueName: \"kubernetes.io/projected/aaa16c33-a697-4a91-b30b-68fb63c959c5-kube-api-access-8t92s\") pod \"maas-controller-575cb5b89b-xd5wg\" (UID: \"aaa16c33-a697-4a91-b30b-68fb63c959c5\") " pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.173217 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.173132    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aaa16c33-a697-4a91-b30b-68fb63c959c5-webhook-cert\") pod \"maas-controller-575cb5b89b-xd5wg\" (UID: \"aaa16c33-a697-4a91-b30b-68fb63c959c5\") " pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.173217 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.173153    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa16c33-a697-4a91-b30b-68fb63c959c5-metrics-certs\") pod \"maas-controller-575cb5b89b-xd5wg\" (UID: \"aaa16c33-a697-4a91-b30b-68fb63c959c5\") " pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.274461 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.274384    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t92s\" (UniqueName: \"kubernetes.io/projected/aaa16c33-a697-4a91-b30b-68fb63c959c5-kube-api-access-8t92s\") pod \"maas-controller-575cb5b89b-xd5wg\" (UID: \"aaa16c33-a697-4a91-b30b-68fb63c959c5\") " pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.274696 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.274470    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aaa16c33-a697-4a91-b30b-68fb63c959c5-webhook-cert\") pod \"maas-controller-575cb5b89b-xd5wg\" (UID: \"aaa16c33-a697-4a91-b30b-68fb63c959c5\") " pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.274696 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.274495    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa16c33-a697-4a91-b30b-68fb63c959c5-metrics-certs\") pod \"maas-controller-575cb5b89b-xd5wg\" (UID: \"aaa16c33-a697-4a91-b30b-68fb63c959c5\") " pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.277271 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.277245    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa16c33-a697-4a91-b30b-68fb63c959c5-metrics-certs\") pod \"maas-controller-575cb5b89b-xd5wg\" (UID: \"aaa16c33-a697-4a91-b30b-68fb63c959c5\") " pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.277403 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.277284    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aaa16c33-a697-4a91-b30b-68fb63c959c5-webhook-cert\") pod \"maas-controller-575cb5b89b-xd5wg\" (UID: \"aaa16c33-a697-4a91-b30b-68fb63c959c5\") " pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.334835 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.334808    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t92s\" (UniqueName: \"kubernetes.io/projected/aaa16c33-a697-4a91-b30b-68fb63c959c5-kube-api-access-8t92s\") pod \"maas-controller-575cb5b89b-xd5wg\" (UID: \"aaa16c33-a697-4a91-b30b-68fb63c959c5\") " pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.358352 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.358322    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:36.397590 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.397354    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-594b9656dc-jtm5p"]
Nov 04 02:02:36.398911 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:02:36.398872    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114eee1c_8a1a_4d02_82b2_b2a08af9f389.slice/crio-d34151be322d96860a2f76f2ea093f53a52695635f53a56e0fb69bb55f279f1c WatchSource:0}: Error finding container d34151be322d96860a2f76f2ea093f53a52695635f53a56e0fb69bb55f279f1c: Status 404 returned error can't find the container with id d34151be322d96860a2f76f2ea093f53a52695635f53a56e0fb69bb55f279f1c
Nov 04 02:02:36.731848 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.731820    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-575cb5b89b-xd5wg"]
Nov 04 02:02:36.733715 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:02:36.733687    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa16c33_a697_4a91_b30b_68fb63c959c5.slice/crio-9d70949cdbf80b975310a0e99c6bc28c3d38dfcd43c8dc2da25703bff6546b85 WatchSource:0}: Error finding container 9d70949cdbf80b975310a0e99c6bc28c3d38dfcd43c8dc2da25703bff6546b85: Status 404 returned error can't find the container with id 9d70949cdbf80b975310a0e99c6bc28c3d38dfcd43c8dc2da25703bff6546b85
Nov 04 02:02:36.813196 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.813168    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-594b9656dc-jtm5p" event={"ID":"114eee1c-8a1a-4d02-82b2-b2a08af9f389","Type":"ContainerStarted","Data":"d34151be322d96860a2f76f2ea093f53a52695635f53a56e0fb69bb55f279f1c"}
Nov 04 02:02:36.814410 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:36.814385    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-575cb5b89b-xd5wg" event={"ID":"aaa16c33-a697-4a91-b30b-68fb63c959c5","Type":"ContainerStarted","Data":"9d70949cdbf80b975310a0e99c6bc28c3d38dfcd43c8dc2da25703bff6546b85"}
Nov 04 02:02:45.289334 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:45.289307    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 04 02:02:45.800615 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:45.800578    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:45.871449 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:45.871407    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-575cb5b89b-xd5wg" event={"ID":"aaa16c33-a697-4a91-b30b-68fb63c959c5","Type":"ContainerStarted","Data":"acfff3b2d1f1923a0d2b0a7582fdf3d3910c9ad3c12f6985a9de498d7288766a"}
Nov 04 02:02:45.871659 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:45.871480    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:45.873135 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:45.873103    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-594b9656dc-jtm5p" event={"ID":"114eee1c-8a1a-4d02-82b2-b2a08af9f389","Type":"ContainerStarted","Data":"fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f"}
Nov 04 02:02:45.873281 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:45.873204    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:46.028220 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:46.028164    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-575cb5b89b-xd5wg" podStartSLOduration=2.526088405 podStartE2EDuration="11.028150742s" totalImagesPullingTime="8.502062337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:02:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:02:46.025097672 +0000 UTC m=+427.830554304" watchObservedRunningTime="2026-11-04 02:02:46.028150742 +0000 UTC m=+427.833607352"
Nov 04 02:02:46.063048 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:46.062967    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:46.148353 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:46.148293    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-594b9656dc-jtm5p" podStartSLOduration=2.305058506 podStartE2EDuration="11.148275007s" totalImagesPullingTime="8.843216501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:02:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:02:46.1445937 +0000 UTC m=+427.950050332" watchObservedRunningTime="2026-11-04 02:02:46.148275007 +0000 UTC m=+427.953731644"
Nov 04 02:02:48.813072 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.813033    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"]
Nov 04 02:02:48.818117 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.818096    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:48.871900 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.871875    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Nov 04 02:02:48.872103 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.871982    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Nov 04 02:02:48.892779 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.892750    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ftmw\" (UniqueName: \"kubernetes.io/projected/97a87324-2f8e-42fb-99cf-13583bb87171-kube-api-access-2ftmw\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:48.892901 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.892802    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97a87324-2f8e-42fb-99cf-13583bb87171-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:48.892942 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.892911    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/97a87324-2f8e-42fb-99cf-13583bb87171-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:48.892942 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.892932    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97a87324-2f8e-42fb-99cf-13583bb87171-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:48.921751 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.921717    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-j6lmd\""
Nov 04 02:02:48.921911 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.921767    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Nov 04 02:02:48.926386 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.926354    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"]
Nov 04 02:02:48.993685 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.993644    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/97a87324-2f8e-42fb-99cf-13583bb87171-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:48.993685 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.993682    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97a87324-2f8e-42fb-99cf-13583bb87171-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:48.993899 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.993742    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ftmw\" (UniqueName: \"kubernetes.io/projected/97a87324-2f8e-42fb-99cf-13583bb87171-kube-api-access-2ftmw\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:48.993899 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:02:48.993778    4512 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Nov 04 02:02:48.993899 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.993805    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97a87324-2f8e-42fb-99cf-13583bb87171-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:48.993899 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:02:48.993873    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97a87324-2f8e-42fb-99cf-13583bb87171-proxy-source podName:97a87324-2f8e-42fb-99cf-13583bb87171 nodeName:}" failed. No retries permitted until 2026-11-04 02:02:49.493848793 +0000 UTC m=+431.299305422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/97a87324-2f8e-42fb-99cf-13583bb87171-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-xb2fb" (UID: "97a87324-2f8e-42fb-99cf-13583bb87171") : configmap "usage-tenancy-proxy-source" not found
Nov 04 02:02:48.993899 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:02:48.993884    4512 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Nov 04 02:02:48.994144 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:02:48.993949    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97a87324-2f8e-42fb-99cf-13583bb87171-tls-cert podName:97a87324-2f8e-42fb-99cf-13583bb87171 nodeName:}" failed. No retries permitted until 2026-11-04 02:02:49.493932739 +0000 UTC m=+431.299389363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/97a87324-2f8e-42fb-99cf-13583bb87171-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-xb2fb" (UID: "97a87324-2f8e-42fb-99cf-13583bb87171") : secret "usage-logs-tenancy-proxy-tls" not found
Nov 04 02:02:48.994493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:48.994475    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97a87324-2f8e-42fb-99cf-13583bb87171-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:49.014241 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:49.014206    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:49.055146 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:49.055109    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ftmw\" (UniqueName: \"kubernetes.io/projected/97a87324-2f8e-42fb-99cf-13583bb87171-kube-api-access-2ftmw\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:49.268834 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:49.268802    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:02:49.498311 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:49.498277    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/97a87324-2f8e-42fb-99cf-13583bb87171-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:49.498311 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:49.498314    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97a87324-2f8e-42fb-99cf-13583bb87171-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:49.499123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:49.499096    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/97a87324-2f8e-42fb-99cf-13583bb87171-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:49.500851 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:49.500828    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97a87324-2f8e-42fb-99cf-13583bb87171-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-xb2fb\" (UID: \"97a87324-2f8e-42fb-99cf-13583bb87171\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:49.730078 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:49.729997    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:02:50.104863 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:50.104834    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"]
Nov 04 02:02:50.105235 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:02:50.105209    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97a87324_2f8e_42fb_99cf_13583bb87171.slice/crio-d40f2290958a5378c6bfca288dce95a5038cbd6dc13e57c6b53d260f35156260 WatchSource:0}: Error finding container d40f2290958a5378c6bfca288dce95a5038cbd6dc13e57c6b53d260f35156260: Status 404 returned error can't find the container with id d40f2290958a5378c6bfca288dce95a5038cbd6dc13e57c6b53d260f35156260
Nov 04 02:02:50.903104 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:50.903066    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb" event={"ID":"97a87324-2f8e-42fb-99cf-13583bb87171","Type":"ContainerStarted","Data":"d40f2290958a5378c6bfca288dce95a5038cbd6dc13e57c6b53d260f35156260"}
Nov 04 02:02:56.777419 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:02:56.777378    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:02:56.879691 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:56.879660    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:56.879860 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:56.879782    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:02:57.149468 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.149376    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-594b9656dc-jtm5p"]
Nov 04 02:02:57.149665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.149630    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-594b9656dc-jtm5p" podUID="114eee1c-8a1a-4d02-82b2-b2a08af9f389" containerName="manager" containerID="cri-o://fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f" gracePeriod=30
Nov 04 02:02:57.435042 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.435011    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:57.474222 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.474188    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-metrics-certs\" (UniqueName: \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-metrics-certs\") pod \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") "
Nov 04 02:02:57.474394 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.474233    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-webhook-cert\" (UniqueName: \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-webhook-cert\") pod \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") "
Nov 04 02:02:57.474394 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.474294    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/114eee1c-8a1a-4d02-82b2-b2a08af9f389-kube-api-access-7ktv5\" (UniqueName: \"kubernetes.io/projected/114eee1c-8a1a-4d02-82b2-b2a08af9f389-kube-api-access-7ktv5\") pod \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\" (UID: \"114eee1c-8a1a-4d02-82b2-b2a08af9f389\") "
Nov 04 02:02:57.476785 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.476755    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/114eee1c-8a1a-4d02-82b2-b2a08af9f389-kube-api-access-7ktv5" pod "114eee1c-8a1a-4d02-82b2-b2a08af9f389" (UID: "114eee1c-8a1a-4d02-82b2-b2a08af9f389"). InnerVolumeSpecName "kube-api-access-7ktv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:02:57.477057 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.476997    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-metrics-certs" pod "114eee1c-8a1a-4d02-82b2-b2a08af9f389" (UID: "114eee1c-8a1a-4d02-82b2-b2a08af9f389"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:02:57.477147 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.477089    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-webhook-cert" pod "114eee1c-8a1a-4d02-82b2-b2a08af9f389" (UID: "114eee1c-8a1a-4d02-82b2-b2a08af9f389"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:02:57.575914 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.575876    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ktv5\" (UniqueName: \"kubernetes.io/projected/114eee1c-8a1a-4d02-82b2-b2a08af9f389-kube-api-access-7ktv5\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:02:57.575914 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.575907    4512 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-metrics-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:02:57.575914 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.575917    4512 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/114eee1c-8a1a-4d02-82b2-b2a08af9f389-webhook-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:02:57.947918 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.947877    4512 generic.go:356] "Generic (PLEG): container finished" podID="114eee1c-8a1a-4d02-82b2-b2a08af9f389" containerID="fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f" exitCode=0
Nov 04 02:02:57.948447 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.947969    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-594b9656dc-jtm5p" event={"ID":"114eee1c-8a1a-4d02-82b2-b2a08af9f389","Type":"ContainerDied","Data":"fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f"}
Nov 04 02:02:57.948447 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.948019    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-594b9656dc-jtm5p" event={"ID":"114eee1c-8a1a-4d02-82b2-b2a08af9f389","Type":"ContainerDied","Data":"d34151be322d96860a2f76f2ea093f53a52695635f53a56e0fb69bb55f279f1c"}
Nov 04 02:02:57.948447 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.948054    4512 scope.go:122] "RemoveContainer" containerID="fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f"
Nov 04 02:02:57.948447 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.948222    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-594b9656dc-jtm5p"
Nov 04 02:02:57.959906 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.959882    4512 scope.go:122] "RemoveContainer" containerID="fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f"
Nov 04 02:02:57.960210 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:02:57.960188    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f\": container with ID starting with fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f not found: ID does not exist" containerID="fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f"
Nov 04 02:02:57.960314 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:57.960217    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f"} err="failed to get container status \"fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f\": rpc error: code = NotFound desc = could not find container \"fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f\": container with ID starting with fa7268b4cb3d283c24c258d7ff2ab50ba9b2e77a8543ae2067fd61c22b64235f not found: ID does not exist"
Nov 04 02:02:58.173692 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:58.173652    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-594b9656dc-jtm5p"]
Nov 04 02:02:58.176847 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:58.176813    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-594b9656dc-jtm5p"]
Nov 04 02:02:58.313561 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:02:58.313525    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="114eee1c-8a1a-4d02-82b2-b2a08af9f389" path="/var/lib/kubelet/pods/114eee1c-8a1a-4d02-82b2-b2a08af9f389/volumes"
Nov 04 02:03:08.011674 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:08.011624    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb" event={"ID":"97a87324-2f8e-42fb-99cf-13583bb87171","Type":"ContainerStarted","Data":"a3ac77a8e1b514e0061ed59321ab58aea7673eaf46e718a0d6a521c5a7703e28"}
Nov 04 02:03:08.012079 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:08.011727    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:03:08.129415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:08.129361    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb" podStartSLOduration=3.143933138 podStartE2EDuration="20.129348105s" totalImagesPullingTime="16.985414967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:08.127314913 +0000 UTC m=+449.932771545" watchObservedRunningTime="2026-11-04 02:03:08.129348105 +0000 UTC m=+449.934804773"
Nov 04 02:03:14.416080 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:14.416039    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:14.667493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:14.667418    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:14.876494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:14.876458    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:15.075600 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.075551    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:15.247334 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.247000    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-67t5h"]
Nov 04 02:03:15.248071 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.248049    4512 cpu_manager.go:414] "containerMap: removing container" podUID="114eee1c-8a1a-4d02-82b2-b2a08af9f389" containerName="manager"
Nov 04 02:03:15.248150 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.248075    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="114eee1c-8a1a-4d02-82b2-b2a08af9f389" containerName="manager"
Nov 04 02:03:15.248258 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.248240    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="114eee1c-8a1a-4d02-82b2-b2a08af9f389" containerName="manager"
Nov 04 02:03:15.253727 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.253706    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h"
Nov 04 02:03:15.280616 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.280586    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:15.308411 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.308377    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-67t5h"]
Nov 04 02:03:15.354708 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.354574    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Nov 04 02:03:15.354893 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.354742    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-d7ln2\""
Nov 04 02:03:15.374678 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.374645    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a712b8d8-6477-411b-ab42-a925a4a2e397-cert\") pod \"kserve-controller-manager-776cc66d47-67t5h\" (UID: \"a712b8d8-6477-411b-ab42-a925a4a2e397\") " pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h"
Nov 04 02:03:15.374845 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.374752    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xd87\" (UniqueName: \"kubernetes.io/projected/a712b8d8-6477-411b-ab42-a925a4a2e397-kube-api-access-5xd87\") pod \"kserve-controller-manager-776cc66d47-67t5h\" (UID: \"a712b8d8-6477-411b-ab42-a925a4a2e397\") " pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h"
Nov 04 02:03:15.476168 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.476135    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a712b8d8-6477-411b-ab42-a925a4a2e397-cert\") pod \"kserve-controller-manager-776cc66d47-67t5h\" (UID: \"a712b8d8-6477-411b-ab42-a925a4a2e397\") " pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h"
Nov 04 02:03:15.476675 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.476195    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xd87\" (UniqueName: \"kubernetes.io/projected/a712b8d8-6477-411b-ab42-a925a4a2e397-kube-api-access-5xd87\") pod \"kserve-controller-manager-776cc66d47-67t5h\" (UID: \"a712b8d8-6477-411b-ab42-a925a4a2e397\") " pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h"
Nov 04 02:03:15.478906 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.478872    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a712b8d8-6477-411b-ab42-a925a4a2e397-cert\") pod \"kserve-controller-manager-776cc66d47-67t5h\" (UID: \"a712b8d8-6477-411b-ab42-a925a4a2e397\") " pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h"
Nov 04 02:03:15.479279 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.479250    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-jvj68"]
Nov 04 02:03:15.482201 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.482184    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-jvj68"
Nov 04 02:03:15.488200 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.488120    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:15.534610 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.534585    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 04 02:03:15.534738 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.534618    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-fv68h\""
Nov 04 02:03:15.537126 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.537101    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xd87\" (UniqueName: \"kubernetes.io/projected/a712b8d8-6477-411b-ab42-a925a4a2e397-kube-api-access-5xd87\") pod \"kserve-controller-manager-776cc66d47-67t5h\" (UID: \"a712b8d8-6477-411b-ab42-a925a4a2e397\") " pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h"
Nov 04 02:03:15.540932 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.540910    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-jvj68"]
Nov 04 02:03:15.568611 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.568590    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h"
Nov 04 02:03:15.576735 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.576705    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cb40c10-cfc5-4d9e-954f-462e1ba04257-tls-certs\") pod \"model-serving-api-75bd89fdfc-jvj68\" (UID: \"9cb40c10-cfc5-4d9e-954f-462e1ba04257\") " pod="opendatahub/model-serving-api-75bd89fdfc-jvj68"
Nov 04 02:03:15.576873 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.576750    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s855p\" (UniqueName: \"kubernetes.io/projected/9cb40c10-cfc5-4d9e-954f-462e1ba04257-kube-api-access-s855p\") pod \"model-serving-api-75bd89fdfc-jvj68\" (UID: \"9cb40c10-cfc5-4d9e-954f-462e1ba04257\") " pod="opendatahub/model-serving-api-75bd89fdfc-jvj68"
Nov 04 02:03:15.678379 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.678341    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cb40c10-cfc5-4d9e-954f-462e1ba04257-tls-certs\") pod \"model-serving-api-75bd89fdfc-jvj68\" (UID: \"9cb40c10-cfc5-4d9e-954f-462e1ba04257\") " pod="opendatahub/model-serving-api-75bd89fdfc-jvj68"
Nov 04 02:03:15.678632 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.678391    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s855p\" (UniqueName: \"kubernetes.io/projected/9cb40c10-cfc5-4d9e-954f-462e1ba04257-kube-api-access-s855p\") pod \"model-serving-api-75bd89fdfc-jvj68\" (UID: \"9cb40c10-cfc5-4d9e-954f-462e1ba04257\") " pod="opendatahub/model-serving-api-75bd89fdfc-jvj68"
Nov 04 02:03:15.681117 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.681087    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cb40c10-cfc5-4d9e-954f-462e1ba04257-tls-certs\") pod \"model-serving-api-75bd89fdfc-jvj68\" (UID: \"9cb40c10-cfc5-4d9e-954f-462e1ba04257\") " pod="opendatahub/model-serving-api-75bd89fdfc-jvj68"
Nov 04 02:03:15.739187 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.739158    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s855p\" (UniqueName: \"kubernetes.io/projected/9cb40c10-cfc5-4d9e-954f-462e1ba04257-kube-api-access-s855p\") pod \"model-serving-api-75bd89fdfc-jvj68\" (UID: \"9cb40c10-cfc5-4d9e-954f-462e1ba04257\") " pod="opendatahub/model-serving-api-75bd89fdfc-jvj68"
Nov 04 02:03:15.798486 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.798444    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-jvj68"
Nov 04 02:03:15.943688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:15.943657    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-67t5h"]
Nov 04 02:03:15.945281 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:03:15.945258    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda712b8d8_6477_411b_ab42_a925a4a2e397.slice/crio-ffee227dc56f5576b24b00c4992fc61ecc0e6f9a720d261599ad73745be91a26 WatchSource:0}: Error finding container ffee227dc56f5576b24b00c4992fc61ecc0e6f9a720d261599ad73745be91a26: Status 404 returned error can't find the container with id ffee227dc56f5576b24b00c4992fc61ecc0e6f9a720d261599ad73745be91a26
Nov 04 02:03:16.058019 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:16.057984    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h" event={"ID":"a712b8d8-6477-411b-ab42-a925a4a2e397","Type":"ContainerStarted","Data":"ffee227dc56f5576b24b00c4992fc61ecc0e6f9a720d261599ad73745be91a26"}
Nov 04 02:03:16.171804 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:16.171778    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-jvj68"]
Nov 04 02:03:16.173155 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:03:16.173122    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb40c10_cfc5_4d9e_954f_462e1ba04257.slice/crio-c74f7db238937634bba9a5ea56cf8b1046f4ac1ad1fa714c29a9989addde573a WatchSource:0}: Error finding container c74f7db238937634bba9a5ea56cf8b1046f4ac1ad1fa714c29a9989addde573a: Status 404 returned error can't find the container with id c74f7db238937634bba9a5ea56cf8b1046f4ac1ad1fa714c29a9989addde573a
Nov 04 02:03:16.709412 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:16.709380    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:17.066361 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:17.066327    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-jvj68" event={"ID":"9cb40c10-cfc5-4d9e-954f-462e1ba04257","Type":"ContainerStarted","Data":"c74f7db238937634bba9a5ea56cf8b1046f4ac1ad1fa714c29a9989addde573a"}
Nov 04 02:03:17.163762 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:17.163711    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:17.381665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:17.381579    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:19.024428 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:19.024391    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb"
Nov 04 02:03:19.353087 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:19.352980    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:19.603819 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:19.603735    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:19.848021 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:19.847979    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:21.949362 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:21.949266    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:22.099978 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:22.099946    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-jvj68" event={"ID":"9cb40c10-cfc5-4d9e-954f-462e1ba04257","Type":"ContainerStarted","Data":"d7f71a4d6bbb7c51d014ab38d98925fe475f468bc0eb498df2a268d75c6f592e"}
Nov 04 02:03:22.100233 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:22.100211    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-jvj68"
Nov 04 02:03:22.151204 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:22.151172    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:22.219093 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:22.218983    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-jvj68" podStartSLOduration=1.8110433879999999 podStartE2EDuration="7.218963457s" totalImagesPullingTime="5.407920069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:22.216709233 +0000 UTC m=+464.022165856" watchObservedRunningTime="2026-11-04 02:03:22.218963457 +0000 UTC m=+464.024420093"
Nov 04 02:03:22.358149 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:22.358112    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:22.564704 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:22.564662    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:22.763106 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:22.763074    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:22.963131 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:22.963098    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:23.107295 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:23.107205    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h" event={"ID":"a712b8d8-6477-411b-ab42-a925a4a2e397","Type":"ContainerStarted","Data":"40b8637de6b863f8d601278649217cdc9b584c243594608e8a08f922d6bf6977"}
Nov 04 02:03:23.107446 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:23.107323    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h"
Nov 04 02:03:23.162073 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:23.162038    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:23.224683 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:23.224629    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h" podStartSLOduration=1.744019872 podStartE2EDuration="8.224614609s" totalImagesPullingTime="6.480594737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:23.22203123 +0000 UTC m=+465.027487862" watchObservedRunningTime="2026-11-04 02:03:23.224614609 +0000 UTC m=+465.030071240"
Nov 04 02:03:26.548108 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:26.548068    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:26.777887 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:03:26.777844    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:03:33.113903 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:33.113865    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-jvj68"
Nov 04 02:03:35.520371 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:35.520336    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:35.726092 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:35.726062    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:36.185156 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.185124    4512 generic.go:356] "Generic (PLEG): container finished" podID="8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2" containerID="c1545af460685b1fd70337ca46f94381594694c7f0115d7f890eb7a1284cfcb9" exitCode=0
Nov 04 02:03:36.185325 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.185197    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj" event={"ID":"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2","Type":"ContainerDied","Data":"c1545af460685b1fd70337ca46f94381594694c7f0115d7f890eb7a1284cfcb9"}
Nov 04 02:03:36.185635 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.185619    4512 scope.go:122] "RemoveContainer" containerID="c1545af460685b1fd70337ca46f94381594694c7f0115d7f890eb7a1284cfcb9"
Nov 04 02:03:36.186815 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.186783    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:03:36.186869 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.186859    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:03:36.186907 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.186892    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:03:36.209726 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.209689    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69f65d44c7-8lctn"]
Nov 04 02:03:36.283097 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.283075    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69f65d44c7-8lctn"
Nov 04 02:03:36.289564 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.289536    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69f65d44c7-8lctn"]
Nov 04 02:03:36.371735 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.371703    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xxnr6\""
Nov 04 02:03:36.402926 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.402890    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpgjz\" (UniqueName: \"kubernetes.io/projected/1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e-kube-api-access-cpgjz\") pod \"authorino-69f65d44c7-8lctn\" (UID: \"1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e\") " pod="kuadrant-system/authorino-69f65d44c7-8lctn"
Nov 04 02:03:36.504459 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.504422    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpgjz\" (UniqueName: \"kubernetes.io/projected/1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e-kube-api-access-cpgjz\") pod \"authorino-69f65d44c7-8lctn\" (UID: \"1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e\") " pod="kuadrant-system/authorino-69f65d44c7-8lctn"
Nov 04 02:03:36.566480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.566445    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpgjz\" (UniqueName: \"kubernetes.io/projected/1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e-kube-api-access-cpgjz\") pod \"authorino-69f65d44c7-8lctn\" (UID: \"1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e\") " pod="kuadrant-system/authorino-69f65d44c7-8lctn"
Nov 04 02:03:36.602557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.602483    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69f65d44c7-8lctn"]
Nov 04 02:03:36.602763 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.602751    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69f65d44c7-8lctn"
Nov 04 02:03:36.822734 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:36.822618    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:03:37.016104 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:03:37.016058    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1eb8b3c0_ce2c_47c5_bcef_8066e87ef42e.slice/crio-1faa0d5b38f6fba52f25069392ca129d2c1b02c0781fc7ec54f46696f03a81bd WatchSource:0}: Error finding container 1faa0d5b38f6fba52f25069392ca129d2c1b02c0781fc7ec54f46696f03a81bd: Status 404 returned error can't find the container with id 1faa0d5b38f6fba52f25069392ca129d2c1b02c0781fc7ec54f46696f03a81bd
Nov 04 02:03:37.018153 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:37.018127    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69f65d44c7-8lctn"]
Nov 04 02:03:37.216860 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:37.216761    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj" event={"ID":"8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2","Type":"ContainerStarted","Data":"5d48450a2155f201848b0da4b579dffaac216cacf42bae5f590f7bda370fe42b"}
Nov 04 02:03:37.218137 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:37.218115    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69f65d44c7-8lctn" event={"ID":"1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e","Type":"ContainerStarted","Data":"1faa0d5b38f6fba52f25069392ca129d2c1b02c0781fc7ec54f46696f03a81bd"}
Nov 04 02:03:37.434016 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:37.433977    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:03:37.435257 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:37.435232    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 02:03:37.435332 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:37.435283    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj" podUID="8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 02:03:38.434246 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:38.434192    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 02:03:38.434658 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:38.434333    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj" podUID="8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 02:03:39.236690 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:39.236646    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-69f65d44c7-8lctn" podUID="1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e" containerName="authorino" containerID="cri-o://c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f" gracePeriod=30
Nov 04 02:03:39.237059 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:39.237029    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69f65d44c7-8lctn" event={"ID":"1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e","Type":"ContainerStarted","Data":"c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f"}
Nov 04 02:03:39.367418 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:39.367361    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-69f65d44c7-8lctn" podStartSLOduration=1.800984862 podStartE2EDuration="3.367346688s" totalImagesPullingTime="1.566361826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:39.366535228 +0000 UTC m=+481.171991861" watchObservedRunningTime="2026-11-04 02:03:39.367346688 +0000 UTC m=+481.172803320"
Nov 04 02:03:39.434525 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:39.434469    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 02:03:39.434910 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:39.434567    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj" podUID="8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 02:03:39.583175 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:39.583142    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69f65d44c7-8lctn"
Nov 04 02:03:39.659123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:39.659085    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e-kube-api-access-cpgjz\" (UniqueName: \"kubernetes.io/projected/1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e-kube-api-access-cpgjz\") pod \"1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e\" (UID: \"1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e\") "
Nov 04 02:03:39.661558 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:39.661500    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e-kube-api-access-cpgjz" pod "1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e" (UID: "1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e"). InnerVolumeSpecName "kube-api-access-cpgjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:03:39.760022 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:39.759972    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpgjz\" (UniqueName: \"kubernetes.io/projected/1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e-kube-api-access-cpgjz\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:03:40.242200 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.242162    4512 generic.go:356] "Generic (PLEG): container finished" podID="1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e" containerID="c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f" exitCode=0
Nov 04 02:03:40.242387 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.242206    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69f65d44c7-8lctn"
Nov 04 02:03:40.242387 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.242248    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69f65d44c7-8lctn" event={"ID":"1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e","Type":"ContainerDied","Data":"c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f"}
Nov 04 02:03:40.242387 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.242296    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69f65d44c7-8lctn" event={"ID":"1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e","Type":"ContainerDied","Data":"1faa0d5b38f6fba52f25069392ca129d2c1b02c0781fc7ec54f46696f03a81bd"}
Nov 04 02:03:40.242387 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.242317    4512 scope.go:122] "RemoveContainer" containerID="c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f"
Nov 04 02:03:40.252291 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.252269    4512 scope.go:122] "RemoveContainer" containerID="c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f"
Nov 04 02:03:40.252600 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:03:40.252580    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f\": container with ID starting with c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f not found: ID does not exist" containerID="c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f"
Nov 04 02:03:40.252654 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.252609    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f"} err="failed to get container status \"c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f\": rpc error: code = NotFound desc = could not find container \"c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f\": container with ID starting with c28635fd317a741f2d01332fb381e63e59bcc7a78efdfa59aa448f22a7c4a16f not found: ID does not exist"
Nov 04 02:03:40.424669 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.424637    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69f65d44c7-8lctn"]
Nov 04 02:03:40.428432 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.428407    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-69f65d44c7-8lctn"]
Nov 04 02:03:40.434197 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.434175    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 02:03:40.434278 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:40.434223    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj" podUID="8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 02:03:41.434580 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:41.434542    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 02:03:41.435003 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:41.434616    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj" podUID="8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 02:03:42.311913 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:42.311875    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e" path="/var/lib/kubelet/pods/1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e/volumes"
Nov 04 02:03:42.437685 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:42.437655    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:03:42.438066 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:42.437920    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:03:42.438719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:42.438702    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj"
Nov 04 02:03:42.755866 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:42.755827    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:42.958916 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:42.958880    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:03:54.117950 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:03:54.117877    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-67t5h"
Nov 04 02:03:56.777543 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:03:56.777479    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:04:25.003533 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:04:25.003477    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:04:26.776976 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:04:26.776939    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:04:26.898456 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:04:26.898414    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:04:56.776956 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:04:56.776913    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:04:57.398610 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:04:57.398575    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:04:57.599276 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:04:57.599232    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:04:57.828621 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:04:57.828580    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:14.752877 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:14.752848    4512 generic.go:356] "Generic (PLEG): container finished" podID="aaa16c33-a697-4a91-b30b-68fb63c959c5" containerID="acfff3b2d1f1923a0d2b0a7582fdf3d3910c9ad3c12f6985a9de498d7288766a" exitCode=1
Nov 04 02:05:14.753225 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:14.752925    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-575cb5b89b-xd5wg" event={"ID":"aaa16c33-a697-4a91-b30b-68fb63c959c5","Type":"ContainerDied","Data":"acfff3b2d1f1923a0d2b0a7582fdf3d3910c9ad3c12f6985a9de498d7288766a"}
Nov 04 02:05:14.753281 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:14.753267    4512 scope.go:122] "RemoveContainer" containerID="acfff3b2d1f1923a0d2b0a7582fdf3d3910c9ad3c12f6985a9de498d7288766a"
Nov 04 02:05:15.760560 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:15.760531    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-575cb5b89b-xd5wg" event={"ID":"aaa16c33-a697-4a91-b30b-68fb63c959c5","Type":"ContainerStarted","Data":"d9e5200f590f6e08116a7320055304ecf18297d92442b44905913f54350fba7d"}
Nov 04 02:05:15.760869 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:15.760746    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:05:26.768131 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:26.768045    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-575cb5b89b-xd5wg"
Nov 04 02:05:26.778670 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:05:26.778638    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:05:27.258748 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:27.258695    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:30.872938 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:30.872904    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:38.529760 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:38.529729    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:05:38.534237 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:38.534212    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:05:40.199052 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:40.199014    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:40.451321 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:40.451237    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:47.234026 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:47.233981    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:47.487898 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:05:47.487802    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:56.777617 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:05:56.777580    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:06:09.387749 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:09.387718    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:09.637737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:09.637699    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:14.621404 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:14.621369    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:15.143913 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:15.143874    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:18.209907 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:18.209863    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:18.462600 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:18.462500    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:25.633549 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:25.633491    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:25.939890 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:25.939809    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:26.777296 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:06:26.777260    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:06:27.621679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:27.621639    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:41.406225 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.406186    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"]
Nov 04 02:06:41.406704 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.406637    4512 cpu_manager.go:414] "containerMap: removing container" podUID="1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e" containerName="authorino"
Nov 04 02:06:41.406704 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.406650    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e" containerName="authorino"
Nov 04 02:06:41.406776 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.406727    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="1eb8b3c0-ce2c-47c5-bcef-8066e87ef42e" containerName="authorino"
Nov 04 02:06:41.409984 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.409966    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.463910 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.463880    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 04 02:06:41.464069 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.463985    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rczvg\""
Nov 04 02:06:41.482217 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.482188    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 04 02:06:41.503864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.503817    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.503864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.503861    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.504095 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.503975    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.504095 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.503999    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.504175 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.504092    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.504175 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.504119    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlxs2\" (UniqueName: \"kubernetes.io/projected/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-kube-api-access-jlxs2\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.513730 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.513700    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Nov 04 02:06:41.516096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.516074    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"]
Nov 04 02:06:41.561423 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.561387    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:41.605164 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.605127    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.605359 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.605180    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.605359 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.605221    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.605359 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.605243    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlxs2\" (UniqueName: \"kubernetes.io/projected/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-kube-api-access-jlxs2\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.605359 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.605269    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.605587 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.605414    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.605721 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.605702    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.605763 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.605735    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.605834 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.605817    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.607667 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.607648    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.607943 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.607925    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.669477 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.669401    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlxs2\" (UniqueName: \"kubernetes.io/projected/6fe316ae-8d04-40f0-a6ca-7c4e71893b86-kube-api-access-jlxs2\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl\" (UID: \"6fe316ae-8d04-40f0-a6ca-7c4e71893b86\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:41.723318 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:41.723279    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:42.103362 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:42.103331    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"]
Nov 04 02:06:42.105027 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:06:42.104996    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe316ae_8d04_40f0_a6ca_7c4e71893b86.slice/crio-f815c670662ea2ff7608f7d4490b19f29e9c01131d9a05a99f7f7d4bfb8b0301 WatchSource:0}: Error finding container f815c670662ea2ff7608f7d4490b19f29e9c01131d9a05a99f7f7d4bfb8b0301: Status 404 returned error can't find the container with id f815c670662ea2ff7608f7d4490b19f29e9c01131d9a05a99f7f7d4bfb8b0301
Nov 04 02:06:42.106395 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:42.106377    4512 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:06:42.228680 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:42.228646    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl" event={"ID":"6fe316ae-8d04-40f0-a6ca-7c4e71893b86","Type":"ContainerStarted","Data":"f815c670662ea2ff7608f7d4490b19f29e9c01131d9a05a99f7f7d4bfb8b0301"}
Nov 04 02:06:45.248912 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:45.248876    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl" event={"ID":"6fe316ae-8d04-40f0-a6ca-7c4e71893b86","Type":"ContainerStarted","Data":"217ee938805b9063f2d7672302a21735040b46c3a72f9982905bc088bcdd4815"}
Nov 04 02:06:45.366640 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:45.366584    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl" podStartSLOduration=2.185333006 podStartE2EDuration="4.366570802s" totalImagesPullingTime="2.181237796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:45.365689567 +0000 UTC m=+667.171146200" watchObservedRunningTime="2026-11-04 02:06:45.366570802 +0000 UTC m=+667.172027434"
Nov 04 02:06:45.724364 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:45.724292    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:50.639576 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:50.639466    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:50.842987 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:50.842935    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:51.724216 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:51.724176    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:51.736830 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:51.736801    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:52.298286 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:52.298246    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl"
Nov 04 02:06:55.106848 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.106810    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"]
Nov 04 02:06:55.110478 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.110456    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.162716 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.162690    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Nov 04 02:06:55.214863 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.214828    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"]
Nov 04 02:06:55.242755 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.242721    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e18368a3-6ffc-481d-a7fe-011065a9a24a-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.242932 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.242772    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.242932 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.242886    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.242932 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.242911    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.243058 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.242938    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.243058 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.243013    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4z8m\" (UniqueName: \"kubernetes.io/projected/e18368a3-6ffc-481d-a7fe-011065a9a24a-kube-api-access-b4z8m\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.256393 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.256357    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:55.343733 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.343698    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e18368a3-6ffc-481d-a7fe-011065a9a24a-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.343949 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.343763    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.343949 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.343928    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.344079 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.343979    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.344079 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.344010    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.344079 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.344043    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4z8m\" (UniqueName: \"kubernetes.io/projected/e18368a3-6ffc-481d-a7fe-011065a9a24a-kube-api-access-b4z8m\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.344276 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.344247    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.344359 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.344340    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.344428 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.344405    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.346381 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.346350    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e18368a3-6ffc-481d-a7fe-011065a9a24a-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.346540 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.346523    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e18368a3-6ffc-481d-a7fe-011065a9a24a-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.405127 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.405047    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4z8m\" (UniqueName: \"kubernetes.io/projected/e18368a3-6ffc-481d-a7fe-011065a9a24a-kube-api-access-b4z8m\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-b77xj\" (UID: \"e18368a3-6ffc-481d-a7fe-011065a9a24a\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.423868 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.423834    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:55.805495 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:55.805463    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"]
Nov 04 02:06:55.807448 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:06:55.807421    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode18368a3_6ffc_481d_a7fe_011065a9a24a.slice/crio-85662bd06673f514dd5cb7971a41c24fa9cbb92820e5641afa2c42a33408e399 WatchSource:0}: Error finding container 85662bd06673f514dd5cb7971a41c24fa9cbb92820e5641afa2c42a33408e399: Status 404 returned error can't find the container with id 85662bd06673f514dd5cb7971a41c24fa9cbb92820e5641afa2c42a33408e399
Nov 04 02:06:56.311794 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:56.311760    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj" event={"ID":"e18368a3-6ffc-481d-a7fe-011065a9a24a","Type":"ContainerStarted","Data":"85662bd06673f514dd5cb7971a41c24fa9cbb92820e5641afa2c42a33408e399"}
Nov 04 02:06:56.776934 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:06:56.776895    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:06:57.319231 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:57.319192    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj" event={"ID":"e18368a3-6ffc-481d-a7fe-011065a9a24a","Type":"ContainerStarted","Data":"7727a4f948494eb007e25f7be4bbd21b828e6758cbb83235145778ef9ed67ae6"}
Nov 04 02:06:57.424921 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:57.424885    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:06:57.438145 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:57.438099    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj" podStartSLOduration=1.857132092 podStartE2EDuration="2.438084546s" totalImagesPullingTime="580.952454ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:57.43654675 +0000 UTC m=+679.242003381" watchObservedRunningTime="2026-11-04 02:06:57.438084546 +0000 UTC m=+679.243541176"
Nov 04 02:06:59.115309 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:06:59.115267    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:02.155782 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:02.155740    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:05.424440 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.424398    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:07:05.448424 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.448383    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:07:05.864843 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.864805    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-k7wrt"]
Nov 04 02:07:05.869039 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.869014    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:05.932935 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.932903    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Nov 04 02:07:05.939548 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.939484    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-k7wrt"]
Nov 04 02:07:05.954021 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.953986    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:05.954021 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.954020    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-home\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:05.954271 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.954127    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b11d4503-32db-4aaf-a651-78fc2b44c413-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:05.954271 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.954185    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbqnj\" (UniqueName: \"kubernetes.io/projected/b11d4503-32db-4aaf-a651-78fc2b44c413-kube-api-access-pbqnj\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:05.954271 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.954217    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:05.954445 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:05.954317    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.035365 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.035297    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:06.055033 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.054999    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.055033 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.055040    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.055260 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.055061    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-home\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.055260 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.055109    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b11d4503-32db-4aaf-a651-78fc2b44c413-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.055260 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.055134    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbqnj\" (UniqueName: \"kubernetes.io/projected/b11d4503-32db-4aaf-a651-78fc2b44c413-kube-api-access-pbqnj\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.055260 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.055156    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.055543 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.055483    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.055672 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.055547    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-home\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.055672 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.055612    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.057679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.057658    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b11d4503-32db-4aaf-a651-78fc2b44c413-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.057764 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.057660    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b11d4503-32db-4aaf-a651-78fc2b44c413-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.117104 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.117019    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbqnj\" (UniqueName: \"kubernetes.io/projected/b11d4503-32db-4aaf-a651-78fc2b44c413-kube-api-access-pbqnj\") pod \"echo-simulator-kserve-64b6f8f96c-k7wrt\" (UID: \"b11d4503-32db-4aaf-a651-78fc2b44c413\") " pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.181679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.181637    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:06.384364 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.384282    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj"
Nov 04 02:07:06.563035 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:06.563003    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-k7wrt"]
Nov 04 02:07:06.600476 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:07:06.600446    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11d4503_32db_4aaf_a651_78fc2b44c413.slice/crio-68a331a18348042c9153250d757650efee393273fe54baaf915ed2a69f4956e9 WatchSource:0}: Error finding container 68a331a18348042c9153250d757650efee393273fe54baaf915ed2a69f4956e9: Status 404 returned error can't find the container with id 68a331a18348042c9153250d757650efee393273fe54baaf915ed2a69f4956e9
Nov 04 02:07:07.381963 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:07.381923    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt" event={"ID":"b11d4503-32db-4aaf-a651-78fc2b44c413","Type":"ContainerStarted","Data":"68a331a18348042c9153250d757650efee393273fe54baaf915ed2a69f4956e9"}
Nov 04 02:07:09.395281 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:09.395187    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt" event={"ID":"b11d4503-32db-4aaf-a651-78fc2b44c413","Type":"ContainerStarted","Data":"8a70c6d8951877362a4540e0e2a2d72d05d599cd32ed5da0f1b28104355e1a70"}
Nov 04 02:07:09.513963 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:09.513896    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt" podStartSLOduration=1.9861466349999999 podStartE2EDuration="4.513881446s" totalImagesPullingTime="2.527734811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:09.511995642 +0000 UTC m=+691.317452273" watchObservedRunningTime="2026-11-04 02:07:09.513881446 +0000 UTC m=+691.319338096"
Nov 04 02:07:10.077931 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.077894    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"]
Nov 04 02:07:10.081849 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.081828    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.177447 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.177418    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Nov 04 02:07:10.182500 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.182472    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:10.186165 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.186143    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"]
Nov 04 02:07:10.195134 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.195111    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c17897b5-2085-4013-aeb3-2b55b6503572-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.195222 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.195155    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.195287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.195263    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdzcf\" (UniqueName: \"kubernetes.io/projected/c17897b5-2085-4013-aeb3-2b55b6503572-kube-api-access-hdzcf\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.195436 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.195415    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.195479 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.195464    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.195569 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.195549    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.211425 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.211398    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:10.296592 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.296547    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdzcf\" (UniqueName: \"kubernetes.io/projected/c17897b5-2085-4013-aeb3-2b55b6503572-kube-api-access-hdzcf\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.296785 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.296661    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.296785 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.296695    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.296785 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.296718    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.296785 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.296779    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c17897b5-2085-4013-aeb3-2b55b6503572-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.296986 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.296808    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.297133 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.297107    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.297199 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.297166    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.297253 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.297191    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.299480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.299454    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c17897b5-2085-4013-aeb3-2b55b6503572-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.299648 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.299629    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c17897b5-2085-4013-aeb3-2b55b6503572-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.357677 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.357601    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdzcf\" (UniqueName: \"kubernetes.io/projected/c17897b5-2085-4013-aeb3-2b55b6503572-kube-api-access-hdzcf\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5\" (UID: \"c17897b5-2085-4013-aeb3-2b55b6503572\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.395691 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.395663    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:10.774626 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:10.774596    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"]
Nov 04 02:07:10.775597 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:07:10.775573    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17897b5_2085_4013_aeb3_2b55b6503572.slice/crio-c9d0f0e7c252c831b1720817bf7e60acac657e7cf13b31caa6c85b355ccdf28c WatchSource:0}: Error finding container c9d0f0e7c252c831b1720817bf7e60acac657e7cf13b31caa6c85b355ccdf28c: Status 404 returned error can't find the container with id c9d0f0e7c252c831b1720817bf7e60acac657e7cf13b31caa6c85b355ccdf28c
Nov 04 02:07:11.408394 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:11.408363    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5" event={"ID":"c17897b5-2085-4013-aeb3-2b55b6503572","Type":"ContainerStarted","Data":"c9d0f0e7c252c831b1720817bf7e60acac657e7cf13b31caa6c85b355ccdf28c"}
Nov 04 02:07:12.415535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:12.415481    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5" event={"ID":"c17897b5-2085-4013-aeb3-2b55b6503572","Type":"ContainerStarted","Data":"67bd7184b72d3fd330641e602bf0262aefc45b8030d9ba1e45aae26eab666f13"}
Nov 04 02:07:12.534322 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:12.534260    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5" podStartSLOduration=1.956732296 podStartE2EDuration="2.534247723s" totalImagesPullingTime="577.515427ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:12.531437041 +0000 UTC m=+694.336893673" watchObservedRunningTime="2026-11-04 02:07:12.534247723 +0000 UTC m=+694.339704394"
Nov 04 02:07:13.396286 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:13.396252    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:15.308690 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:15.308633    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:16.181840 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:16.181811    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:16.194081 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:16.194047    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:16.454723 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:16.454640    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-k7wrt"
Nov 04 02:07:20.396178 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:20.396145    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:20.408069 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:20.408045    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:20.478874 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:20.478850    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5"
Nov 04 02:07:26.777803 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:07:26.777736    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:07:27.967883 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:27.967840    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:37.315342 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.315307    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"]
Nov 04 02:07:37.319306 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.319285    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.424196 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.424149    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"]
Nov 04 02:07:37.452619 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.452580    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.452827 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.452640    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.452827 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.452730    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn97w\" (UniqueName: \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-kube-api-access-cn97w\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.452827 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.452805    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.453002 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.452845    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.453002 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.452863    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.453002 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.452972    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.453164 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.453005    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.453164 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.453063    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.554481 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.554446    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.554481 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.554485    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.554753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.554538    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.554753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.554568    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.554753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.554701    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.555013 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.554991    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.555141 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.554994    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.555141 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.555070    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.555141 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.555118    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn97w\" (UniqueName: \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-kube-api-access-cn97w\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.555300 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.555211    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.555300 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.555273    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.555604 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.555578    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.555707 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.555658    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.555769 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.555713    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.557212 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.557183    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.557329 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.557211    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.616034 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.615957    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.616161 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.616064    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn97w\" (UniqueName: \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-kube-api-access-cn97w\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:37.632804 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:37.632773    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:38.060138 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:38.060108    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"]
Nov 04 02:07:38.062159 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:07:38.062132    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a2fa07_309f_4e8b_a6e6_a0c8849b9b7a.slice/crio-a0b79366c7fdacbbd26244353a3966f65e71f7f4b1be7b0a385a4d63f4c172a6 WatchSource:0}: Error finding container a0b79366c7fdacbbd26244353a3966f65e71f7f4b1be7b0a385a4d63f4c172a6: Status 404 returned error can't find the container with id a0b79366c7fdacbbd26244353a3966f65e71f7f4b1be7b0a385a4d63f4c172a6
Nov 04 02:07:38.064235 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:38.064202    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:07:38.064286 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:38.064272    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:07:38.064323 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:38.064302    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:07:38.569561 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:38.569499    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" event={"ID":"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a","Type":"ContainerStarted","Data":"0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038"}
Nov 04 02:07:38.569935 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:38.569603    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" event={"ID":"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a","Type":"ContainerStarted","Data":"a0b79366c7fdacbbd26244353a3966f65e71f7f4b1be7b0a385a4d63f4c172a6"}
Nov 04 02:07:38.632909 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:38.632879    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:38.712849 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:38.712793    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" podStartSLOduration=1.712777781 podStartE2EDuration="1.712777781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:38.710799842 +0000 UTC m=+720.516256491" watchObservedRunningTime="2026-11-04 02:07:38.712777781 +0000 UTC m=+720.518234411"
Nov 04 02:07:39.634752 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:39.634714    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:07:39.635125 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:39.634773    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 02:07:40.633699 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:40.633667    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:07:40.633855 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:40.633726    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 02:07:41.633845 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:41.633810    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:07:41.634284 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:41.633864    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 02:07:42.633295 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:42.633256    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:07:42.633475 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:42.633318    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 02:07:43.634033 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:43.633996    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:07:43.634468 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:43.634054    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 02:07:44.633405 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:44.633374    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 02:07:44.633601 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:44.633427    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 02:07:46.406447 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:46.406402    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:46.406965 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:46.406784    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:46.407450 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:46.407427    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:07:46.787005 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:46.786966    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-86c994f77b-qwzgv"]
Nov 04 02:07:46.794456 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:46.794430    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86c994f77b-qwzgv"
Nov 04 02:07:46.846622 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:46.846573    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xxnr6\""
Nov 04 02:07:46.846622 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:46.846573    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 04 02:07:46.847424 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:46.847403    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86c994f77b-qwzgv"]
Nov 04 02:07:46.970965 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:46.970918    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0c66053c-2382-4412-93ca-f67f1274fd08-tls-cert\") pod \"authorino-86c994f77b-qwzgv\" (UID: \"0c66053c-2382-4412-93ca-f67f1274fd08\") " pod="kuadrant-system/authorino-86c994f77b-qwzgv"
Nov 04 02:07:46.971134 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:46.970978    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nw8j\" (UniqueName: \"kubernetes.io/projected/0c66053c-2382-4412-93ca-f67f1274fd08-kube-api-access-7nw8j\") pod \"authorino-86c994f77b-qwzgv\" (UID: \"0c66053c-2382-4412-93ca-f67f1274fd08\") " pod="kuadrant-system/authorino-86c994f77b-qwzgv"
Nov 04 02:07:47.072107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:47.072017    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0c66053c-2382-4412-93ca-f67f1274fd08-tls-cert\") pod \"authorino-86c994f77b-qwzgv\" (UID: \"0c66053c-2382-4412-93ca-f67f1274fd08\") " pod="kuadrant-system/authorino-86c994f77b-qwzgv"
Nov 04 02:07:47.072107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:47.072065    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nw8j\" (UniqueName: \"kubernetes.io/projected/0c66053c-2382-4412-93ca-f67f1274fd08-kube-api-access-7nw8j\") pod \"authorino-86c994f77b-qwzgv\" (UID: \"0c66053c-2382-4412-93ca-f67f1274fd08\") " pod="kuadrant-system/authorino-86c994f77b-qwzgv"
Nov 04 02:07:47.074572 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:47.074555    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0c66053c-2382-4412-93ca-f67f1274fd08-tls-cert\") pod \"authorino-86c994f77b-qwzgv\" (UID: \"0c66053c-2382-4412-93ca-f67f1274fd08\") " pod="kuadrant-system/authorino-86c994f77b-qwzgv"
Nov 04 02:07:47.133056 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:47.133019    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nw8j\" (UniqueName: \"kubernetes.io/projected/0c66053c-2382-4412-93ca-f67f1274fd08-kube-api-access-7nw8j\") pod \"authorino-86c994f77b-qwzgv\" (UID: \"0c66053c-2382-4412-93ca-f67f1274fd08\") " pod="kuadrant-system/authorino-86c994f77b-qwzgv"
Nov 04 02:07:47.407115 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:47.407036    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86c994f77b-qwzgv"
Nov 04 02:07:47.470258 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:47.470214    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:47.786037 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:47.785983    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:47.787761 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:47.787728    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86c994f77b-qwzgv"]
Nov 04 02:07:47.788921 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:07:47.788896    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c66053c_2382_4412_93ca_f67f1274fd08.slice/crio-bd3fff3e0fcb350dbd61294531b8b4ecddc589521808ef9a6aa1dc05d6b7f06a WatchSource:0}: Error finding container bd3fff3e0fcb350dbd61294531b8b4ecddc589521808ef9a6aa1dc05d6b7f06a: Status 404 returned error can't find the container with id bd3fff3e0fcb350dbd61294531b8b4ecddc589521808ef9a6aa1dc05d6b7f06a
Nov 04 02:07:48.649171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:48.649135    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86c994f77b-qwzgv" event={"ID":"0c66053c-2382-4412-93ca-f67f1274fd08","Type":"ContainerStarted","Data":"bd3fff3e0fcb350dbd61294531b8b4ecddc589521808ef9a6aa1dc05d6b7f06a"}
Nov 04 02:07:49.092746 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:49.092709    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:49.343039 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:49.342977    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:49.656062 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:49.655982    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86c994f77b-qwzgv" event={"ID":"0c66053c-2382-4412-93ca-f67f1274fd08","Type":"ContainerStarted","Data":"7168494e0bc3108d23469e15a947c8551ca7168edc0d4dd95e735db757cd4019"}
Nov 04 02:07:49.772100 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:07:49.772047    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-86c994f77b-qwzgv" podStartSLOduration=2.236153441 podStartE2EDuration="3.77203329s" totalImagesPullingTime="1.535879849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:49.770438326 +0000 UTC m=+731.575894959" watchObservedRunningTime="2026-11-04 02:07:49.77203329 +0000 UTC m=+731.577489921"
Nov 04 02:07:56.777907 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:07:56.777870    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:08:26.777226 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:08:26.777139    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:08:28.309755 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:28.309716    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:44.621784 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:44.621745    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:44.834234 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:44.834198    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:45.254935 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:45.254884    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:45.503188 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:45.503154    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:45.714320 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:45.714221    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:45.921366 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:45.921298    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:46.125960 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:46.125927    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:46.782638 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:46.782605    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:48.091259 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:48.091212    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:48.341324 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:48.341234    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:49.405899 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:49.405859    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:49.611091 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:49.611055    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:50.113585 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:50.113550    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:50.333031 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:50.332989    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:50.748374 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:50.748340    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:50.953801 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:50.953758    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:52.337810 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:52.337777    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:52.538731 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:52.538691    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:52.814296 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:52.814260    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:53.051209 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.051176    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:53.264430 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.264387    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:53.409342 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.409301    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"]
Nov 04 02:08:53.413051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.413023    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:08:53.467351 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.467323    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-29e7e8-dockercfg-qqwh8\""
Nov 04 02:08:53.470004 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.469972    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:53.517006 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.516882    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-29e7e8\""
Nov 04 02:08:53.520144 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.520117    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"]
Nov 04 02:08:53.542344 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.542298    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"]
Nov 04 02:08:53.542753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.542726    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x\" (UID: \"a5c9cdc8-5445-40db-8f1e-ae6b61861eca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:08:53.542967 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.542942    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5plqz\" (UniqueName: \"kubernetes.io/projected/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-kube-api-access-5plqz\") pod \"payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x\" (UID: \"a5c9cdc8-5445-40db-8f1e-ae6b61861eca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:08:53.546217 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.546195    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:08:53.603951 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.603920    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"]
Nov 04 02:08:53.643844 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.643809    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x\" (UID: \"a5c9cdc8-5445-40db-8f1e-ae6b61861eca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:08:53.644015 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.643854    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kvfr\" (UniqueName: \"kubernetes.io/projected/7c2a1484-282c-49d8-b4d6-43b41d23f18a-kube-api-access-5kvfr\") pod \"payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn\" (UID: \"7c2a1484-282c-49d8-b4d6-43b41d23f18a\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:08:53.644015 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.643905    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a1484-282c-49d8-b4d6-43b41d23f18a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn\" (UID: \"7c2a1484-282c-49d8-b4d6-43b41d23f18a\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:08:53.644015 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.643942    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5plqz\" (UniqueName: \"kubernetes.io/projected/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-kube-api-access-5plqz\") pod \"payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x\" (UID: \"a5c9cdc8-5445-40db-8f1e-ae6b61861eca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:08:53.644585 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.644564    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x\" (UID: \"a5c9cdc8-5445-40db-8f1e-ae6b61861eca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:08:53.693775 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.693734    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:53.704897 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.704860    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5plqz\" (UniqueName: \"kubernetes.io/projected/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-kube-api-access-5plqz\") pod \"payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x\" (UID: \"a5c9cdc8-5445-40db-8f1e-ae6b61861eca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:08:53.727545 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.727491    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:08:53.745362 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.745331    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a1484-282c-49d8-b4d6-43b41d23f18a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn\" (UID: \"7c2a1484-282c-49d8-b4d6-43b41d23f18a\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:08:53.745560 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.745492    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kvfr\" (UniqueName: \"kubernetes.io/projected/7c2a1484-282c-49d8-b4d6-43b41d23f18a-kube-api-access-5kvfr\") pod \"payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn\" (UID: \"7c2a1484-282c-49d8-b4d6-43b41d23f18a\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:08:53.746820 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.746787    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a1484-282c-49d8-b4d6-43b41d23f18a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn\" (UID: \"7c2a1484-282c-49d8-b4d6-43b41d23f18a\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:08:53.806366 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.806333    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kvfr\" (UniqueName: \"kubernetes.io/projected/7c2a1484-282c-49d8-b4d6-43b41d23f18a-kube-api-access-5kvfr\") pod \"payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn\" (UID: \"7c2a1484-282c-49d8-b4d6-43b41d23f18a\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:08:53.860634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:53.860598    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:08:54.121058 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:54.121031    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"]
Nov 04 02:08:54.122476 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:08:54.122448    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c9cdc8_5445_40db_8f1e_ae6b61861eca.slice/crio-b65fac4f126f3d566d672c148530b3191e9630a6899013f7cbae1077b2bbc9b4 WatchSource:0}: Error finding container b65fac4f126f3d566d672c148530b3191e9630a6899013f7cbae1077b2bbc9b4: Status 404 returned error can't find the container with id b65fac4f126f3d566d672c148530b3191e9630a6899013f7cbae1077b2bbc9b4
Nov 04 02:08:54.243327 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:54.243300    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"]
Nov 04 02:08:54.245139 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:08:54.245113    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c2a1484_282c_49d8_b4d6_43b41d23f18a.slice/crio-65895400b02671e3e63809c875f80fa12e8e06b234b0cfcccd46b855dd5d34b7 WatchSource:0}: Error finding container 65895400b02671e3e63809c875f80fa12e8e06b234b0cfcccd46b855dd5d34b7: Status 404 returned error can't find the container with id 65895400b02671e3e63809c875f80fa12e8e06b234b0cfcccd46b855dd5d34b7
Nov 04 02:08:55.031763 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:55.031725    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn" event={"ID":"7c2a1484-282c-49d8-b4d6-43b41d23f18a","Type":"ContainerStarted","Data":"65895400b02671e3e63809c875f80fa12e8e06b234b0cfcccd46b855dd5d34b7"}
Nov 04 02:08:55.033026 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:55.033001    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x" event={"ID":"a5c9cdc8-5445-40db-8f1e-ae6b61861eca","Type":"ContainerStarted","Data":"b65fac4f126f3d566d672c148530b3191e9630a6899013f7cbae1077b2bbc9b4"}
Nov 04 02:08:56.166762 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:56.159474    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:56.362049 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:56.362013    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:56.564148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:56.564109    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:56.776573 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:56.776533    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:56.803327 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:08:56.803274    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:08:57.997439 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:57.997381    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:58.068441 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.068394    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn" event={"ID":"7c2a1484-282c-49d8-b4d6-43b41d23f18a","Type":"ContainerStarted","Data":"cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658"}
Nov 04 02:08:58.068665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.068564    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:08:58.070163 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.070138    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x" event={"ID":"a5c9cdc8-5445-40db-8f1e-ae6b61861eca","Type":"ContainerStarted","Data":"60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2"}
Nov 04 02:08:58.070298 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.070281    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:08:58.189558 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.189481    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn" podStartSLOduration=2.21605507 podStartE2EDuration="5.18946646s" totalImagesPullingTime="2.97341139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:08:58.187110321 +0000 UTC m=+799.992566949" watchObservedRunningTime="2026-11-04 02:08:58.18946646 +0000 UTC m=+799.994923089"
Nov 04 02:08:58.214856 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.214825    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:58.305307 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.305193    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x" podStartSLOduration=2.289939957 podStartE2EDuration="5.305175897s" totalImagesPullingTime="3.01523594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:08:58.304128081 +0000 UTC m=+800.109584713" watchObservedRunningTime="2026-11-04 02:08:58.305175897 +0000 UTC m=+800.110632530"
Nov 04 02:08:58.369093 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.369055    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"]
Nov 04 02:08:58.372946 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.372930    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:08:58.426943 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.426878    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-3bf869\""
Nov 04 02:08:58.427290 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.427267    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-3bf869-dockercfg-jcs45\""
Nov 04 02:08:58.430442 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.430418    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"]
Nov 04 02:08:58.501970 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.501931    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6227bc3-6dcb-46e4-b990-07c9ad43252a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j\" (UID: \"f6227bc3-6dcb-46e4-b990-07c9ad43252a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:08:58.502141 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.501982    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6pvf\" (UniqueName: \"kubernetes.io/projected/f6227bc3-6dcb-46e4-b990-07c9ad43252a-kube-api-access-t6pvf\") pod \"payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j\" (UID: \"f6227bc3-6dcb-46e4-b990-07c9ad43252a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:08:58.507109 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.507079    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"]
Nov 04 02:08:58.511204 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.511183    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:08:58.568998 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.568917    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"]
Nov 04 02:08:58.603395 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.603362    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6227bc3-6dcb-46e4-b990-07c9ad43252a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j\" (UID: \"f6227bc3-6dcb-46e4-b990-07c9ad43252a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:08:58.603649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.603415    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6pvf\" (UniqueName: \"kubernetes.io/projected/f6227bc3-6dcb-46e4-b990-07c9ad43252a-kube-api-access-t6pvf\") pod \"payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j\" (UID: \"f6227bc3-6dcb-46e4-b990-07c9ad43252a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:08:58.603649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.603466    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvp4c\" (UniqueName: \"kubernetes.io/projected/891a1e83-0481-4fdf-bc83-91c61b411557-kube-api-access-xvp4c\") pod \"payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h\" (UID: \"891a1e83-0481-4fdf-bc83-91c61b411557\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:08:58.603649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.603565    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/891a1e83-0481-4fdf-bc83-91c61b411557-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h\" (UID: \"891a1e83-0481-4fdf-bc83-91c61b411557\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:08:58.604204 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.604183    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6227bc3-6dcb-46e4-b990-07c9ad43252a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j\" (UID: \"f6227bc3-6dcb-46e4-b990-07c9ad43252a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:08:58.664465 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.664430    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6pvf\" (UniqueName: \"kubernetes.io/projected/f6227bc3-6dcb-46e4-b990-07c9ad43252a-kube-api-access-t6pvf\") pod \"payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j\" (UID: \"f6227bc3-6dcb-46e4-b990-07c9ad43252a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:08:58.687097 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.687062    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:08:58.704564 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.704531    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvp4c\" (UniqueName: \"kubernetes.io/projected/891a1e83-0481-4fdf-bc83-91c61b411557-kube-api-access-xvp4c\") pod \"payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h\" (UID: \"891a1e83-0481-4fdf-bc83-91c61b411557\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:08:58.704685 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.704618    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/891a1e83-0481-4fdf-bc83-91c61b411557-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h\" (UID: \"891a1e83-0481-4fdf-bc83-91c61b411557\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:08:58.705438 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.705387    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/891a1e83-0481-4fdf-bc83-91c61b411557-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h\" (UID: \"891a1e83-0481-4fdf-bc83-91c61b411557\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:08:58.765654 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.765627    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvp4c\" (UniqueName: \"kubernetes.io/projected/891a1e83-0481-4fdf-bc83-91c61b411557-kube-api-access-xvp4c\") pod \"payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h\" (UID: \"891a1e83-0481-4fdf-bc83-91c61b411557\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:08:58.825079 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:58.824993    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:08:59.062072 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:59.062047    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"]
Nov 04 02:08:59.063011 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:08:59.062988    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6227bc3_6dcb_46e4_b990_07c9ad43252a.slice/crio-dc32f3a17f1c799616aea01476e20bc6aa8ead95dcc5609e3366d6c795b51b31 WatchSource:0}: Error finding container dc32f3a17f1c799616aea01476e20bc6aa8ead95dcc5609e3366d6c795b51b31: Status 404 returned error can't find the container with id dc32f3a17f1c799616aea01476e20bc6aa8ead95dcc5609e3366d6c795b51b31
Nov 04 02:08:59.096307 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:59.096268    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j" event={"ID":"f6227bc3-6dcb-46e4-b990-07c9ad43252a","Type":"ContainerStarted","Data":"dc32f3a17f1c799616aea01476e20bc6aa8ead95dcc5609e3366d6c795b51b31"}
Nov 04 02:08:59.208886 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:08:59.208857    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"]
Nov 04 02:08:59.209654 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:08:59.209631    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod891a1e83_0481_4fdf_bc83_91c61b411557.slice/crio-9574c14a92643427f1703b7b8b1e9a3415e538b7842044c4467c79086bb25b1c WatchSource:0}: Error finding container 9574c14a92643427f1703b7b8b1e9a3415e538b7842044c4467c79086bb25b1c: Status 404 returned error can't find the container with id 9574c14a92643427f1703b7b8b1e9a3415e538b7842044c4467c79086bb25b1c
Nov 04 02:09:00.104579 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.104539    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j" event={"ID":"f6227bc3-6dcb-46e4-b990-07c9ad43252a","Type":"ContainerStarted","Data":"d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892"}
Nov 04 02:09:00.105001 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.104710    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:09:00.106300 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.106265    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h" event={"ID":"891a1e83-0481-4fdf-bc83-91c61b411557","Type":"ContainerStarted","Data":"9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec"}
Nov 04 02:09:00.106439 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.106318    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h" event={"ID":"891a1e83-0481-4fdf-bc83-91c61b411557","Type":"ContainerStarted","Data":"9574c14a92643427f1703b7b8b1e9a3415e538b7842044c4467c79086bb25b1c"}
Nov 04 02:09:00.106439 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.106339    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:09:00.223321 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.223271    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j" podStartSLOduration=2.223258523 podStartE2EDuration="2.223258523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:00.21963388 +0000 UTC m=+802.025090511" watchObservedRunningTime="2026-11-04 02:09:00.223258523 +0000 UTC m=+802.028715153"
Nov 04 02:09:00.336704 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.336647    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h" podStartSLOduration=2.336633776 podStartE2EDuration="2.336633776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:00.334619568 +0000 UTC m=+802.140076199" watchObservedRunningTime="2026-11-04 02:09:00.336633776 +0000 UTC m=+802.142090449"
Nov 04 02:09:00.530927 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.530888    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:00.734482 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.734438    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:00.884240 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.884154    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"]
Nov 04 02:09:00.888035 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.888012    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:09:00.942978 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.942939    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-196e7e\""
Nov 04 02:09:00.943176 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.942995    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-196e7e-dockercfg-ngrqd\""
Nov 04 02:09:00.946879 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:00.946856    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"]
Nov 04 02:09:01.029762 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.029721    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm76z\" (UniqueName: \"kubernetes.io/projected/c7b95100-975a-48f8-b092-678cd9d7168e-kube-api-access-xm76z\") pod \"payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn\" (UID: \"c7b95100-975a-48f8-b092-678cd9d7168e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:09:01.030036 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.029783    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b95100-975a-48f8-b092-678cd9d7168e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn\" (UID: \"c7b95100-975a-48f8-b092-678cd9d7168e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:09:01.033168 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.033137    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"]
Nov 04 02:09:01.037056 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.037026    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:09:01.095872 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.095837    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"]
Nov 04 02:09:01.131148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.131112    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm76z\" (UniqueName: \"kubernetes.io/projected/c7b95100-975a-48f8-b092-678cd9d7168e-kube-api-access-xm76z\") pod \"payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn\" (UID: \"c7b95100-975a-48f8-b092-678cd9d7168e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:09:01.131684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.131171    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b95100-975a-48f8-b092-678cd9d7168e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn\" (UID: \"c7b95100-975a-48f8-b092-678cd9d7168e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:09:01.131684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.131211    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aba8252-af93-4aa7-985f-d4164191b6c8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb\" (UID: \"0aba8252-af93-4aa7-985f-d4164191b6c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:09:01.131684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.131282    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6s9d\" (UniqueName: \"kubernetes.io/projected/0aba8252-af93-4aa7-985f-d4164191b6c8-kube-api-access-p6s9d\") pod \"payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb\" (UID: \"0aba8252-af93-4aa7-985f-d4164191b6c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:09:01.132059 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.132036    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b95100-975a-48f8-b092-678cd9d7168e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn\" (UID: \"c7b95100-975a-48f8-b092-678cd9d7168e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:09:01.192242 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.192171    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm76z\" (UniqueName: \"kubernetes.io/projected/c7b95100-975a-48f8-b092-678cd9d7168e-kube-api-access-xm76z\") pod \"payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn\" (UID: \"c7b95100-975a-48f8-b092-678cd9d7168e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:09:01.203097 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.203066    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:09:01.233582 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.232868    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aba8252-af93-4aa7-985f-d4164191b6c8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb\" (UID: \"0aba8252-af93-4aa7-985f-d4164191b6c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:09:01.233582 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.232975    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6s9d\" (UniqueName: \"kubernetes.io/projected/0aba8252-af93-4aa7-985f-d4164191b6c8-kube-api-access-p6s9d\") pod \"payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb\" (UID: \"0aba8252-af93-4aa7-985f-d4164191b6c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:09:01.233861 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.233646    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aba8252-af93-4aa7-985f-d4164191b6c8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb\" (UID: \"0aba8252-af93-4aa7-985f-d4164191b6c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:09:01.294555 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.294479    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6s9d\" (UniqueName: \"kubernetes.io/projected/0aba8252-af93-4aa7-985f-d4164191b6c8-kube-api-access-p6s9d\") pod \"payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb\" (UID: \"0aba8252-af93-4aa7-985f-d4164191b6c8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:09:01.351284 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.351243    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:09:01.469760 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.469682    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:01.582776 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.582750    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"]
Nov 04 02:09:01.584610 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:09:01.584581    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b95100_975a_48f8_b092_678cd9d7168e.slice/crio-00a0e7bae8ed36b108dc7dfd87d9312c9d449b30643cbf74b1d7a3a726ef21ee WatchSource:0}: Error finding container 00a0e7bae8ed36b108dc7dfd87d9312c9d449b30643cbf74b1d7a3a726ef21ee: Status 404 returned error can't find the container with id 00a0e7bae8ed36b108dc7dfd87d9312c9d449b30643cbf74b1d7a3a726ef21ee
Nov 04 02:09:01.738988 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.738961    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"]
Nov 04 02:09:01.740914 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:09:01.740883    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aba8252_af93_4aa7_985f_d4164191b6c8.slice/crio-260b9e5e04d5238102aa8ac241579df5b2f0a30d174925caf9871328e609c18d WatchSource:0}: Error finding container 260b9e5e04d5238102aa8ac241579df5b2f0a30d174925caf9871328e609c18d: Status 404 returned error can't find the container with id 260b9e5e04d5238102aa8ac241579df5b2f0a30d174925caf9871328e609c18d
Nov 04 02:09:01.742003 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:01.741864    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:02.120538 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:02.120399    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn" event={"ID":"c7b95100-975a-48f8-b092-678cd9d7168e","Type":"ContainerStarted","Data":"7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8"}
Nov 04 02:09:02.120728 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:02.120534    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn" event={"ID":"c7b95100-975a-48f8-b092-678cd9d7168e","Type":"ContainerStarted","Data":"00a0e7bae8ed36b108dc7dfd87d9312c9d449b30643cbf74b1d7a3a726ef21ee"}
Nov 04 02:09:02.120728 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:02.120569    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:09:02.122153 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:02.122125    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb" event={"ID":"0aba8252-af93-4aa7-985f-d4164191b6c8","Type":"ContainerStarted","Data":"b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a"}
Nov 04 02:09:02.122467 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:02.122163    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb" event={"ID":"0aba8252-af93-4aa7-985f-d4164191b6c8","Type":"ContainerStarted","Data":"260b9e5e04d5238102aa8ac241579df5b2f0a30d174925caf9871328e609c18d"}
Nov 04 02:09:02.122467 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:02.122260    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:09:02.238243 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:02.238188    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn" podStartSLOduration=2.238174328 podStartE2EDuration="2.238174328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:02.235973325 +0000 UTC m=+804.041429956" watchObservedRunningTime="2026-11-04 02:09:02.238174328 +0000 UTC m=+804.043630958"
Nov 04 02:09:02.354994 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:02.354941    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb" podStartSLOduration=2.354927769 podStartE2EDuration="2.354927769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:02.352843768 +0000 UTC m=+804.158300397" watchObservedRunningTime="2026-11-04 02:09:02.354927769 +0000 UTC m=+804.160384442"
Nov 04 02:09:03.117441 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.117405    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:03.321303 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.321263    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:03.467860 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.467777    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"]
Nov 04 02:09:03.472026 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.472003    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:09:03.529202 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.529166    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"]
Nov 04 02:09:03.558134 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.558099    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/989b004f-6a28-4e46-9bd4-97e6d27df64f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh\" (UID: \"989b004f-6a28-4e46-9bd4-97e6d27df64f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:09:03.558346 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.558207    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr7zv\" (UniqueName: \"kubernetes.io/projected/989b004f-6a28-4e46-9bd4-97e6d27df64f-kube-api-access-wr7zv\") pod \"payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh\" (UID: \"989b004f-6a28-4e46-9bd4-97e6d27df64f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:09:03.575194 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.575146    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-24bea9\""
Nov 04 02:09:03.575775 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.575745    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-24bea9-dockercfg-2jtz8\""
Nov 04 02:09:03.626398 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.626355    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"]
Nov 04 02:09:03.630363 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.630342    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:09:03.659120 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.659084    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/989b004f-6a28-4e46-9bd4-97e6d27df64f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh\" (UID: \"989b004f-6a28-4e46-9bd4-97e6d27df64f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:09:03.659285 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.659148    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr7zv\" (UniqueName: \"kubernetes.io/projected/989b004f-6a28-4e46-9bd4-97e6d27df64f-kube-api-access-wr7zv\") pod \"payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh\" (UID: \"989b004f-6a28-4e46-9bd4-97e6d27df64f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:09:03.659799 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.659777    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/989b004f-6a28-4e46-9bd4-97e6d27df64f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh\" (UID: \"989b004f-6a28-4e46-9bd4-97e6d27df64f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:09:03.688483 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.688451    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"]
Nov 04 02:09:03.720164 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.720091    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr7zv\" (UniqueName: \"kubernetes.io/projected/989b004f-6a28-4e46-9bd4-97e6d27df64f-kube-api-access-wr7zv\") pod \"payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh\" (UID: \"989b004f-6a28-4e46-9bd4-97e6d27df64f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:09:03.760487 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.760453    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7ks8\" (UniqueName: \"kubernetes.io/projected/485907b4-566a-48a7-a669-d3a293bb0d29-kube-api-access-q7ks8\") pod \"payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg\" (UID: \"485907b4-566a-48a7-a669-d3a293bb0d29\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:09:03.760717 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.760498    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/485907b4-566a-48a7-a669-d3a293bb0d29-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg\" (UID: \"485907b4-566a-48a7-a669-d3a293bb0d29\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:09:03.786400 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.786350    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:09:03.861555 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.861497    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7ks8\" (UniqueName: \"kubernetes.io/projected/485907b4-566a-48a7-a669-d3a293bb0d29-kube-api-access-q7ks8\") pod \"payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg\" (UID: \"485907b4-566a-48a7-a669-d3a293bb0d29\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:09:03.861732 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.861591    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/485907b4-566a-48a7-a669-d3a293bb0d29-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg\" (UID: \"485907b4-566a-48a7-a669-d3a293bb0d29\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:09:03.862483 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.862446    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/485907b4-566a-48a7-a669-d3a293bb0d29-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg\" (UID: \"485907b4-566a-48a7-a669-d3a293bb0d29\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:09:03.922417 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.922389    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7ks8\" (UniqueName: \"kubernetes.io/projected/485907b4-566a-48a7-a669-d3a293bb0d29-kube-api-access-q7ks8\") pod \"payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg\" (UID: \"485907b4-566a-48a7-a669-d3a293bb0d29\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:09:03.945180 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:03.945137    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:09:04.161889 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:04.161859    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"]
Nov 04 02:09:04.162987 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:09:04.162961    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989b004f_6a28_4e46_9bd4_97e6d27df64f.slice/crio-c9bcb6a291fa0c8054db8b2898c11639d2080a024d0257b1074acd7d7088d94a WatchSource:0}: Error finding container c9bcb6a291fa0c8054db8b2898c11639d2080a024d0257b1074acd7d7088d94a: Status 404 returned error can't find the container with id c9bcb6a291fa0c8054db8b2898c11639d2080a024d0257b1074acd7d7088d94a
Nov 04 02:09:04.338793 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:09:04.338761    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485907b4_566a_48a7_a669_d3a293bb0d29.slice/crio-e64016a564a915264e4755d6390c8b127d6c18c1d568b449c63384773a9a6ebe WatchSource:0}: Error finding container e64016a564a915264e4755d6390c8b127d6c18c1d568b449c63384773a9a6ebe: Status 404 returned error can't find the container with id e64016a564a915264e4755d6390c8b127d6c18c1d568b449c63384773a9a6ebe
Nov 04 02:09:04.339484 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:04.339459    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"]
Nov 04 02:09:05.143673 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:05.143632    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg" event={"ID":"485907b4-566a-48a7-a669-d3a293bb0d29","Type":"ContainerStarted","Data":"368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3"}
Nov 04 02:09:05.143865 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:05.143695    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg" event={"ID":"485907b4-566a-48a7-a669-d3a293bb0d29","Type":"ContainerStarted","Data":"e64016a564a915264e4755d6390c8b127d6c18c1d568b449c63384773a9a6ebe"}
Nov 04 02:09:05.143865 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:05.143836    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:09:05.145560 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:05.145535    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh" event={"ID":"989b004f-6a28-4e46-9bd4-97e6d27df64f","Type":"ContainerStarted","Data":"56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1"}
Nov 04 02:09:05.145704 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:05.145579    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh" event={"ID":"989b004f-6a28-4e46-9bd4-97e6d27df64f","Type":"ContainerStarted","Data":"c9bcb6a291fa0c8054db8b2898c11639d2080a024d0257b1074acd7d7088d94a"}
Nov 04 02:09:05.145704 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:05.145635    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:09:05.262649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:05.262590    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg" podStartSLOduration=2.262575804 podStartE2EDuration="2.262575804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:05.261098232 +0000 UTC m=+807.066554863" watchObservedRunningTime="2026-11-04 02:09:05.262575804 +0000 UTC m=+807.068032434"
Nov 04 02:09:05.394438 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:05.394327    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh" podStartSLOduration=2.394313427 podStartE2EDuration="2.394313427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:05.392391712 +0000 UTC m=+807.197848343" watchObservedRunningTime="2026-11-04 02:09:05.394313427 +0000 UTC m=+807.199770036"
Nov 04 02:09:07.026190 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:07.026156    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:07.281842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:07.281742    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:09.100122 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:09.100084    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:09:09.100805 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:09.100155    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:09:10.455489 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.455446    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:10.707681 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.707594    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:10.815337 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.815301    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"]
Nov 04 02:09:10.819396 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.819380    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:09:10.877414 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.877382    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-90ad0f\""
Nov 04 02:09:10.879395 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.879364    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"]
Nov 04 02:09:10.925565 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.925533    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-90ad0f-dockercfg-h42hl\""
Nov 04 02:09:10.933962 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.933933    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b45e578-cd49-44ad-8b21-b2e82147c4a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd\" (UID: \"6b45e578-cd49-44ad-8b21-b2e82147c4a6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:09:10.934096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.934001    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmb2v\" (UniqueName: \"kubernetes.io/projected/6b45e578-cd49-44ad-8b21-b2e82147c4a6-kube-api-access-rmb2v\") pod \"payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd\" (UID: \"6b45e578-cd49-44ad-8b21-b2e82147c4a6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:09:10.977876 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.977790    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"]
Nov 04 02:09:10.982842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:10.982785    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:09:11.035042 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.035006    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b45e578-cd49-44ad-8b21-b2e82147c4a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd\" (UID: \"6b45e578-cd49-44ad-8b21-b2e82147c4a6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:09:11.035249 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.035118    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmb2v\" (UniqueName: \"kubernetes.io/projected/6b45e578-cd49-44ad-8b21-b2e82147c4a6-kube-api-access-rmb2v\") pod \"payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd\" (UID: \"6b45e578-cd49-44ad-8b21-b2e82147c4a6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:09:11.036039 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.035968    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b45e578-cd49-44ad-8b21-b2e82147c4a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd\" (UID: \"6b45e578-cd49-44ad-8b21-b2e82147c4a6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:09:11.039829 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.039801    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"]
Nov 04 02:09:11.099493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.099451    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmb2v\" (UniqueName: \"kubernetes.io/projected/6b45e578-cd49-44ad-8b21-b2e82147c4a6-kube-api-access-rmb2v\") pod \"payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd\" (UID: \"6b45e578-cd49-44ad-8b21-b2e82147c4a6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:09:11.115305 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.115273    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:09:11.115496 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.115360    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:09:11.133673 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.133646    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:09:11.136110 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.136087    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwb4r\" (UniqueName: \"kubernetes.io/projected/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-kube-api-access-vwb4r\") pod \"payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24\" (UID: \"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:09:11.136193 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.136130    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24\" (UID: \"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:09:11.237029 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.236986    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwb4r\" (UniqueName: \"kubernetes.io/projected/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-kube-api-access-vwb4r\") pod \"payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24\" (UID: \"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:09:11.237224 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.237074    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24\" (UID: \"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:09:11.237817 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.237796    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24\" (UID: \"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:09:11.298053 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.298020    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwb4r\" (UniqueName: \"kubernetes.io/projected/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-kube-api-access-vwb4r\") pod \"payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24\" (UID: \"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:09:11.301101 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.301071    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:09:11.511997 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.511968    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"]
Nov 04 02:09:11.512931 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:09:11.512908    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b45e578_cd49_44ad_8b21_b2e82147c4a6.slice/crio-3370d4eeedbb8595dc8941700bbff3017eba675b3105ca03add0929e6c12dbf9 WatchSource:0}: Error finding container 3370d4eeedbb8595dc8941700bbff3017eba675b3105ca03add0929e6c12dbf9: Status 404 returned error can't find the container with id 3370d4eeedbb8595dc8941700bbff3017eba675b3105ca03add0929e6c12dbf9
Nov 04 02:09:11.678967 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.678934    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"]
Nov 04 02:09:11.680372 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:09:11.680340    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd99bf0f_215a_43b1_921d_cf1fc6cb4adb.slice/crio-d965bed02fced0c7cd66131d84984f2163026effc282957c1892bd1444b75d11 WatchSource:0}: Error finding container d965bed02fced0c7cd66131d84984f2163026effc282957c1892bd1444b75d11: Status 404 returned error can't find the container with id d965bed02fced0c7cd66131d84984f2163026effc282957c1892bd1444b75d11
Nov 04 02:09:11.931776 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:11.931675    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:12.185348 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:12.185254    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:12.199749 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:12.199713    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24" event={"ID":"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb","Type":"ContainerStarted","Data":"a628b9beba2fc0b9866fc9707948c78de090d4c44a71e3608c26cf957491d432"}
Nov 04 02:09:12.199958 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:12.199915    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:09:12.200041 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:12.199979    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24" event={"ID":"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb","Type":"ContainerStarted","Data":"d965bed02fced0c7cd66131d84984f2163026effc282957c1892bd1444b75d11"}
Nov 04 02:09:12.201671 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:12.201641    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd" event={"ID":"6b45e578-cd49-44ad-8b21-b2e82147c4a6","Type":"ContainerStarted","Data":"47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683"}
Nov 04 02:09:12.201822 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:12.201686    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd" event={"ID":"6b45e578-cd49-44ad-8b21-b2e82147c4a6","Type":"ContainerStarted","Data":"3370d4eeedbb8595dc8941700bbff3017eba675b3105ca03add0929e6c12dbf9"}
Nov 04 02:09:12.201822 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:12.201714    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:09:12.315097 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:12.315020    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24" podStartSLOduration=2.3150057090000002 podStartE2EDuration="2.315005709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:12.314377252 +0000 UTC m=+814.119833883" watchObservedRunningTime="2026-11-04 02:09:12.315005709 +0000 UTC m=+814.120462340"
Nov 04 02:09:12.387755 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:12.387713    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:12.432432 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:12.432380    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd" podStartSLOduration=2.432364997 podStartE2EDuration="2.432364997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:12.430164423 +0000 UTC m=+814.235621071" watchObservedRunningTime="2026-11-04 02:09:12.432364997 +0000 UTC m=+814.237821627"
Nov 04 02:09:13.131946 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:13.131914    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:09:13.132413 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:13.131987    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:09:16.154894 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:16.154867    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:09:16.155268 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:16.154929    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:09:16.972085 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:16.972045    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:21.809019 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:21.808986    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:22.087779 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:22.087287    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:23.211799 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:23.211766    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:09:23.212250 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:23.211833    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:09:26.779230 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:09:26.779190    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:09:26.875236 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:26.875197    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:28.719902 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:28.719868    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:31.764324 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:31.764280    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:34.875756 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:34.875714    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:35.469099 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:35.469065    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:35.671014 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:35.670959    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:37.069770 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:37.069731    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:39.004068 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:39.004003    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:40.885645 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:40.885608    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:41.464588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:41.464548    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:41.670452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:41.670400    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:42.483487 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:42.483446    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:48.001553 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:48.001478    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:48.207017 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:48.206964    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:48.625975 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:48.625939    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:48.828681 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:48.828645    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:50.188099 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:50.188059    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:50.820104 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:50.820070    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:51.025490 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:51.025454    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:53.299413 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:53.299375    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:56.779058 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:09:56.779021    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:09:58.261354 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:58.261316    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:58.465074 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:58.465040    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:58.956022 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:58.955984    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:59.154816 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:09:59.154780    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:03.582759 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:03.582713    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:04.662605 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.662574    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"]
Nov 04 02:10:04.666705 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.666685    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.738119 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.738086    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-home\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.738283 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.738145    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwjkd\" (UniqueName: \"kubernetes.io/projected/d93d5ffe-05d3-444c-a816-87c0822c0ce4-kube-api-access-qwjkd\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.738283 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.738236    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-model-cache\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.738283 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.738268    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tls-certs\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.738475 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.738448    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-dshm\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.738553 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.738535    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tmp-dir\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.769969 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.769931    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"kube-root-ca.crt\""
Nov 04 02:10:04.770171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.770093    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"openshift-service-ca.crt\""
Nov 04 02:10:04.770383 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.770337    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"simulator-w4-196e7e-kserve-self-signed-certs\""
Nov 04 02:10:04.770383 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.770372    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"default-dockercfg-758fc\""
Nov 04 02:10:04.773080 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.773056    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"]
Nov 04 02:10:04.813957 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.813905    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:04.839177 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.839141    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-home\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.839357 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.839195    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwjkd\" (UniqueName: \"kubernetes.io/projected/d93d5ffe-05d3-444c-a816-87c0822c0ce4-kube-api-access-qwjkd\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.839357 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.839218    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-model-cache\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.839357 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.839236    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tls-certs\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.839357 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.839307    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-dshm\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.839357 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.839332    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tmp-dir\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.839650 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.839625    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-home\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.839687 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.839658    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-model-cache\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.839777 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.839753    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tmp-dir\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.841790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.841773    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-dshm\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.841915 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.841898    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tls-certs\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.900431 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.900403    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwjkd\" (UniqueName: \"kubernetes.io/projected/d93d5ffe-05d3-444c-a816-87c0822c0ce4-kube-api-access-qwjkd\") pod \"simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") " pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:04.979231 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:04.979145    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:05.357677 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:05.357644    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"]
Nov 04 02:10:05.358584 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:10:05.358556    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd93d5ffe_05d3_444c_a816_87c0822c0ce4.slice/crio-4adf35bc391e65487d026b03e28d0e48bdc0f79cfe91e1a05d5babcf55889d83 WatchSource:0}: Error finding container 4adf35bc391e65487d026b03e28d0e48bdc0f79cfe91e1a05d5babcf55889d83: Status 404 returned error can't find the container with id 4adf35bc391e65487d026b03e28d0e48bdc0f79cfe91e1a05d5babcf55889d83
Nov 04 02:10:05.544867 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:05.544836    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r" event={"ID":"d93d5ffe-05d3-444c-a816-87c0822c0ce4","Type":"ContainerStarted","Data":"681cb37b7f865e9f309db1f8127413fd039c4ce36e7663cf1acdb2af832c6bbf"}
Nov 04 02:10:05.545035 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:05.544875    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r" event={"ID":"d93d5ffe-05d3-444c-a816-87c0822c0ce4","Type":"ContainerStarted","Data":"4adf35bc391e65487d026b03e28d0e48bdc0f79cfe91e1a05d5babcf55889d83"}
Nov 04 02:10:05.663590 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:05.663468    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r" podStartSLOduration=1.663443076 podStartE2EDuration="1.663443076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:05.660677691 +0000 UTC m=+867.466134322" watchObservedRunningTime="2026-11-04 02:10:05.663443076 +0000 UTC m=+867.468899708"
Nov 04 02:10:05.979319 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:05.979234    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:07.987415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:07.987374    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:08.485327 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:08.485278    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:13.383468 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:13.383406    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:14.979284 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:14.979242    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:14.991963 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:14.991936    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:15.627036 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:15.627004    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:10:18.288079 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:18.288043    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:19.823649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:19.823613    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:23.187567 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:23.187496    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:25.360058 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:25.360020    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:26.778165 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:10:26.778127    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:10:28.061046 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:28.061007    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:29.078099 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:29.078061    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:29.358011 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:29.357925    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:32.903044 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:32.902995    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:37.715612 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:37.715565    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:38.578298 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:38.578255    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:10:38.584877 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:38.584851    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:10:41.870646 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:41.870607    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:42.365665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:42.365617    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:42.672576 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:42.672483    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:47.628650 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:47.628611    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:52.476277 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:52.476228    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:56.778225 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:10:56.778187    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:10:57.399699 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:10:57.399658    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:02.291081 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:02.291044    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:03.952218 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:03.952176    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:07.274958 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:07.274923    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:12.190149 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:12.190103    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:17.100557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:17.100489    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:22.064091 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:22.063996    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:26.109942 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:26.109898    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:26.779420 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:11:26.779379    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:11:26.981612 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:26.981574    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:29.421040 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:29.420997    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:31.870304 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:31.870256    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:33.821684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:33.821648    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:34.326165 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.326129    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"]
Nov 04 02:11:34.331368 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.331343    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.384101 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.384066    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-24bea9-openshift-default-dockercfg-g5z8q\""
Nov 04 02:11:34.395302 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.395266    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.395500 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.395312    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m8rg\" (UniqueName: \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-kube-api-access-6m8rg\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.395500 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.395397    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.395500 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.395439    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.395708 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.395545    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.395708 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.395603    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.395708 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.395625    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.395864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.395785    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.395864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.395852    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.438748 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.438711    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"]
Nov 04 02:11:34.496905 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.496868    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497128 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.496933    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497128 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.496970    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497128 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.496993    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497128 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.497025    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497128 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.497080    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497396 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.497163    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497672 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.497645    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m8rg\" (UniqueName: \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-kube-api-access-6m8rg\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497859 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.497838    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.497890    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.497365    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.497764    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.497941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.497554    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.498086 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.497951    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.499561 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.499535    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.500437 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.500415    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.558095 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.558055    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.558247 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.558206    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m8rg\" (UniqueName: \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-kube-api-access-6m8rg\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:34.647143 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:34.647048    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:35.079499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:35.079471    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"]
Nov 04 02:11:35.080325 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:11:35.080291    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c7e0fd5_eece_41b5_94e0_a25a09f28332.slice/crio-9e09453544fa8b5439389837fb580821ee00219d4b30c2d0d6e63f109fb5b5ae WatchSource:0}: Error finding container 9e09453544fa8b5439389837fb580821ee00219d4b30c2d0d6e63f109fb5b5ae: Status 404 returned error can't find the container with id 9e09453544fa8b5439389837fb580821ee00219d4b30c2d0d6e63f109fb5b5ae
Nov 04 02:11:35.082666 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:35.082630    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:11:35.082736 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:35.082717    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:11:35.082779 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:35.082745    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:11:35.174973 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:35.174934    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" event={"ID":"5c7e0fd5-eece-41b5-94e0-a25a09f28332","Type":"ContainerStarted","Data":"5008dab405e9209988b7d6b5f2d1f65c342e15858b5e0e01e25aa12e84cbd437"}
Nov 04 02:11:35.175151 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:35.175001    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" event={"ID":"5c7e0fd5-eece-41b5-94e0-a25a09f28332","Type":"ContainerStarted","Data":"9e09453544fa8b5439389837fb580821ee00219d4b30c2d0d6e63f109fb5b5ae"}
Nov 04 02:11:35.299852 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:35.299785    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" podStartSLOduration=1.299763333 podStartE2EDuration="1.299763333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:11:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:11:35.297671709 +0000 UTC m=+957.103128358" watchObservedRunningTime="2026-11-04 02:11:35.299763333 +0000 UTC m=+957.105219967"
Nov 04 02:11:35.649655 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:35.649612    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:36.649587 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:36.649547    4512 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:11:36.650039 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:36.649621    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 02:11:37.113868 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:37.113829    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:37.416006 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:37.415737    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:37.648219 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:37.648181    4512 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:11:37.648390 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:37.648242    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 02:11:38.648408 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:38.648372    4512 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:11:38.648839 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:38.648429    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 02:11:39.647691 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:39.647646    4512 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:11:39.647913 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:39.647722    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 02:11:40.648541 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:40.648472    4512 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:11:40.648969 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:40.648576    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 02:11:41.647482 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:41.647443    4512 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 02:11:41.647681 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:41.647532    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 02:11:41.701162 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:41.701113    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:42.652363 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:42.652330    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:42.652811 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:42.652791    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:42.653477 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:42.653459    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:11:46.587020 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:46.586984    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:51.490686 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:51.490654    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:54.780977 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.780941    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"]
Nov 04 02:11:54.784888 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.784871    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.833084 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.833049    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.833300 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.833096    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.833300 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.833236    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.833451 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.833312    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.833451 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.833363    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.833602 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.833498    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w64rb\" (UniqueName: \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-kube-api-access-w64rb\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.833655 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.833596    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.833655 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.833646    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.833768 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.833733    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.837126 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.837108    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-90ad0f-openshift-default-dockercfg-nz7vm\""
Nov 04 02:11:54.891164 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.891126    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"]
Nov 04 02:11:54.934624 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.934581    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.934810 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.934635    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.934810 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.934693    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.934810 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.934727    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.934810 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.934752    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.934810 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.934794    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.935023 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.934854    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.935023 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.934902    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.935023 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.934948    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w64rb\" (UniqueName: \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-kube-api-access-w64rb\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.935127 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.935065    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.935230 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.935128    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.935366 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.935344    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.935467 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.935393    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.935467 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.935418    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.937580 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.937553    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.937721 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.937691    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.996117 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.996087    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:54.996295 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:54.996221    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w64rb\" (UniqueName: \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-kube-api-access-w64rb\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:55.099250 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:55.099171    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:55.530480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:55.530437    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"]
Nov 04 02:11:55.530649 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:11:55.530563    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052e4f45_bd16_4ef9_9c17_10a331df3ad4.slice/crio-b2a56256f71b83faa36310c294e84d543b6a48dc6386e622bc2ab9cd04bb2d7d WatchSource:0}: Error finding container b2a56256f71b83faa36310c294e84d543b6a48dc6386e622bc2ab9cd04bb2d7d: Status 404 returned error can't find the container with id b2a56256f71b83faa36310c294e84d543b6a48dc6386e622bc2ab9cd04bb2d7d
Nov 04 02:11:55.533557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:55.533499    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:11:55.533679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:55.533603    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:11:55.533679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:55.533642    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:11:56.368589 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:56.368536    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" event={"ID":"052e4f45-bd16-4ef9-9c17-10a331df3ad4","Type":"ContainerStarted","Data":"302c7287f172a1dfedfe4a9d40bfcf70e8781f04747895b08f01b0903a9971ed"}
Nov 04 02:11:56.369225 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:56.368609    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" event={"ID":"052e4f45-bd16-4ef9-9c17-10a331df3ad4","Type":"ContainerStarted","Data":"b2a56256f71b83faa36310c294e84d543b6a48dc6386e622bc2ab9cd04bb2d7d"}
Nov 04 02:11:56.385644 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:56.385612    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:56.538578 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:56.538482    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" podStartSLOduration=2.538468819 podStartE2EDuration="2.538468819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:11:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:11:56.535423648 +0000 UTC m=+978.340880275" watchObservedRunningTime="2026-11-04 02:11:56.538468819 +0000 UTC m=+978.343925450"
Nov 04 02:11:56.778167 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:11:56.778134    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:11:57.104945 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:57.104859    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:11:57.109152 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:57.108605    4512 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:11:57.109152 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:57.108679    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 02:11:58.105532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:58.104903    4512 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:11:58.105532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:58.104985    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 02:11:59.099865 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:59.099831    4512 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:11:59.101009 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:11:59.100978    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 02:12:00.111534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:00.102869    4512 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:12:00.111534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:00.102941    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 02:12:01.109530 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:01.108879    4512 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:12:01.109530 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:01.108947    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 02:12:01.413635 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:01.413533    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:02.099829 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:02.099784    4512 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 02:12:02.100002 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:02.099854    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 02:12:03.469224 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:03.469185    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:12:03.469993 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:03.469972    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:12:03.470650 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:03.470629    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:12:06.284959 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:06.284920    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:11.175442 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:11.175409    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:16.079438 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:16.079395    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:21.091296 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:21.091259    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:21.528243 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:21.528202    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:25.973010 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:25.972967    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:26.184107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:26.184074    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:26.777893 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:12:26.777857    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:12:26.996327 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:26.996299    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:29.774357 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:29.774319    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:30.795558 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:30.795520    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:31.001414 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:31.001340    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:35.780305 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:35.780271    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:37.316569 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:37.316515    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"]
Nov 04 02:12:37.317067 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:37.316770    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerName="istio-proxy" containerID="cri-o://0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038" gracePeriod=30
Nov 04 02:12:40.599577 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:40.599537    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:41.875913 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:41.875878    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:42.090560 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.090529    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:42.575149 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.575123    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:12:42.690370 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690304    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-socket\") pod \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") "
Nov 04 02:12:42.690538 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690366    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-kube-api-access-cn97w\" (UniqueName: \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-kube-api-access-cn97w\") pod \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") "
Nov 04 02:12:42.690538 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690391    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-data\") pod \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") "
Nov 04 02:12:42.690538 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690419    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-token\" (UniqueName: \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-token\") pod \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") "
Nov 04 02:12:42.690718 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690545    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-podinfo\") pod \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") "
Nov 04 02:12:42.690718 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690596    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-credential-socket\") pod \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") "
Nov 04 02:12:42.690718 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690591    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-socket" pod "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" (UID: "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:12:42.690718 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690677    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-certs\") pod \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") "
Nov 04 02:12:42.690934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690724    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istiod-ca-cert\") pod \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") "
Nov 04 02:12:42.690934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690729    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-data" pod "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" (UID: "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:12:42.690934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690807    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-envoy\") pod \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\" (UID: \"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a\") "
Nov 04 02:12:42.691075 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690946    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-credential-socket" pod "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" (UID: "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:12:42.691075 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.690963    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-certs" pod "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" (UID: "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:12:42.691267 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.691247    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istiod-ca-cert" pod "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" (UID: "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:12:42.691335 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.691269    4512 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-credential-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.691335 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.691290    4512 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.691335 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.691304    4512 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-workload-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.691335 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.691317    4512 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-data\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.693348 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.693324    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-podinfo" pod "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" (UID: "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:12:42.693454 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.693349    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-kube-api-access-cn97w" pod "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" (UID: "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a"). InnerVolumeSpecName "kube-api-access-cn97w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:12:42.693454 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.693379    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-envoy" pod "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" (UID: "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:12:42.693454 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.693405    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-token" pod "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" (UID: "15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:12:42.711874 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.711842    4512 generic.go:356] "Generic (PLEG): container finished" podID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerID="0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038" exitCode=0
Nov 04 02:12:42.711996 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.711881    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"
Nov 04 02:12:42.711996 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.711922    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" event={"ID":"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a","Type":"ContainerDied","Data":"0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038"}
Nov 04 02:12:42.711996 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.711984    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf" event={"ID":"15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a","Type":"ContainerDied","Data":"a0b79366c7fdacbbd26244353a3966f65e71f7f4b1be7b0a385a4d63f4c172a6"}
Nov 04 02:12:42.712158 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.712028    4512 scope.go:122] "RemoveContainer" containerID="0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038"
Nov 04 02:12:42.724098 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.724083    4512 scope.go:122] "RemoveContainer" containerID="0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038"
Nov 04 02:12:42.724339 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:12:42.724321    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038\": container with ID starting with 0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038 not found: ID does not exist" containerID="0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038"
Nov 04 02:12:42.724407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.724352    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038"} err="failed to get container status \"0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038\": rpc error: code = NotFound desc = could not find container \"0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038\": container with ID starting with 0ae98cf34ea2e026ad3a06dc55b1294eb296693fc6a17909726a7c48ef42c038 not found: ID does not exist"
Nov 04 02:12:42.792431 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.792400    4512 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istiod-ca-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.792431 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.792425    4512 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-envoy\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.792431 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.792436    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn97w\" (UniqueName: \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-kube-api-access-cn97w\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.792754 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.792446    4512 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-token\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.792754 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.792455    4512 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a-istio-podinfo\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:12:42.881165 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.881131    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"]
Nov 04 02:12:42.889176 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:42.889149    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf"]
Nov 04 02:12:44.311900 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:44.311872    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" path="/var/lib/kubelet/pods/15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a/volumes"
Nov 04 02:12:45.573350 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:45.573315    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:47.094494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.094456    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"]
Nov 04 02:12:47.095250 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.095226    4512 cpu_manager.go:414] "containerMap: removing container" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerName="istio-proxy"
Nov 04 02:12:47.095343 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.095252    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerName="istio-proxy"
Nov 04 02:12:47.095422 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.095409    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="15a2fa07-309f-4e8b-a6e6-a0c8849b9b7a" containerName="istio-proxy"
Nov 04 02:12:47.100238 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.100211    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.152992 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.152960    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-29e7e8-openshift-default-dockercfg-7ntvz\""
Nov 04 02:12:47.207545 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.207498    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"]
Nov 04 02:12:47.237661 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.237635    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.237790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.237670    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c1f6ba3-41f4-4d64-8873-6642f101d280-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.237790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.237711    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djdf8\" (UniqueName: \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-kube-api-access-djdf8\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.237891 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.237866    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.237943 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.237925    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.237978 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.237968    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.238010 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.237992    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.238115 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.238102    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.238148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.238133    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339133 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339108    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339133 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339142    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339327 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339159    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339327 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339209    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339327 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339238    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339327 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339297    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339614 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339329    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c1f6ba3-41f4-4d64-8873-6642f101d280-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339614 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339456    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djdf8\" (UniqueName: \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-kube-api-access-djdf8\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339614 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339572    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339773 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339610    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339773 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339665    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339773 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339709    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.339917 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.339873    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.340055 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.340020    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c1f6ba3-41f4-4d64-8873-6642f101d280-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.341829 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.341811    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.341908 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.341891    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.400581 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.400520    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.401112 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.401092    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djdf8\" (UniqueName: \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-kube-api-access-djdf8\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.417844 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.417824    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:47.845383 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.845345    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"]
Nov 04 02:12:47.845820 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:12:47.845791    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c1f6ba3_41f4_4d64_8873_6642f101d280.slice/crio-37710b81f99ce305e56766c1355eecf0ffdb6928926243aace34debd1cb63fe4 WatchSource:0}: Error finding container 37710b81f99ce305e56766c1355eecf0ffdb6928926243aace34debd1cb63fe4: Status 404 returned error can't find the container with id 37710b81f99ce305e56766c1355eecf0ffdb6928926243aace34debd1cb63fe4
Nov 04 02:12:47.847868 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.847836    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:12:47.847945 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.847900    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:12:47.847945 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:47.847933    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:12:48.752055 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:48.752021    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" event={"ID":"3c1f6ba3-41f4-4d64-8873-6642f101d280","Type":"ContainerStarted","Data":"894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876"}
Nov 04 02:12:48.752448 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:48.752069    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" event={"ID":"3c1f6ba3-41f4-4d64-8873-6642f101d280","Type":"ContainerStarted","Data":"37710b81f99ce305e56766c1355eecf0ffdb6928926243aace34debd1cb63fe4"}
Nov 04 02:12:48.962364 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:48.962261    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" podStartSLOduration=1.962248544 podStartE2EDuration="1.962248544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:12:48.925647262 +0000 UTC m=+1030.731103896" watchObservedRunningTime="2026-11-04 02:12:48.962248544 +0000 UTC m=+1030.767705174"
Nov 04 02:12:49.418972 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:49.418937    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:49.420705 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:49.420679    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:12:49.420832 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:49.420725    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 02:12:50.418161 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:50.418125    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:12:50.418617 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:50.418173    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 02:12:50.537642 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:50.537598    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:51.418781 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:51.418744    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:12:51.419163 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:51.418796    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 02:12:52.282258 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:52.282221    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:52.419086 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:52.419051    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:12:52.419535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:52.419101    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 02:12:52.512533 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:52.504860    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:53.418428 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:53.418382    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 02:12:53.418725 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:53.418467    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 02:12:54.422115 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:54.422088    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:54.422570 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:54.422548    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:54.423283 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:54.423263    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:12:55.491309 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:55.491277    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:56.777815 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:12:56.777776    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:12:59.063156 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:12:59.063120    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:00.317252 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:00.317215    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:05.176620 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:05.176581    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:10.065949 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:10.065908    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:14.910663 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:14.910626    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:19.860746 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:19.860708    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:24.701261 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:24.701220    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:26.777526 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:26.777479    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:13:29.501759 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:29.501722    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:30.143020 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:30.142980    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:30.387980 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:30.387938    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:34.182567 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:34.182530    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:34.388991 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:34.388950    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:34.910064 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:34.910030    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7348-17efe5"]
Nov 04 02:13:34.913858 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:34.913842    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7348-17efe5"
Nov 04 02:13:34.969814 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:34.969780    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7348-17efe5"]
Nov 04 02:13:34.972873 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:34.972844    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv2ft\" (UniqueName: \"kubernetes.io/projected/13f214d8-4eea-4fd1-87de-955d2a24b5c8-kube-api-access-jv2ft\") pod \"test-curl-7348-17efe5\" (UID: \"13f214d8-4eea-4fd1-87de-955d2a24b5c8\") " pod="opendatahub/test-curl-7348-17efe5"
Nov 04 02:13:35.015916 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:35.015885    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gn5nw\""
Nov 04 02:13:35.074574 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:35.074537    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv2ft\" (UniqueName: \"kubernetes.io/projected/13f214d8-4eea-4fd1-87de-955d2a24b5c8-kube-api-access-jv2ft\") pod \"test-curl-7348-17efe5\" (UID: \"13f214d8-4eea-4fd1-87de-955d2a24b5c8\") " pod="opendatahub/test-curl-7348-17efe5"
Nov 04 02:13:35.135363 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:35.135330    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv2ft\" (UniqueName: \"kubernetes.io/projected/13f214d8-4eea-4fd1-87de-955d2a24b5c8-kube-api-access-jv2ft\") pod \"test-curl-7348-17efe5\" (UID: \"13f214d8-4eea-4fd1-87de-955d2a24b5c8\") " pod="opendatahub/test-curl-7348-17efe5"
Nov 04 02:13:35.227758 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:35.227677    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7348-17efe5"
Nov 04 02:13:35.602076 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:35.602047    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7348-17efe5"]
Nov 04 02:13:35.603275 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:13:35.603247    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13f214d8_4eea_4fd1_87de_955d2a24b5c8.slice/crio-38e0f2b0eafb9a854702be40f12827f0718b88ed8d27f42d67f45226332c01ab WatchSource:0}: Error finding container 38e0f2b0eafb9a854702be40f12827f0718b88ed8d27f42d67f45226332c01ab: Status 404 returned error can't find the container with id 38e0f2b0eafb9a854702be40f12827f0718b88ed8d27f42d67f45226332c01ab
Nov 04 02:13:35.605175 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:35.605157    4512 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:13:36.074782 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:36.074741    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7348-17efe5" event={"ID":"13f214d8-4eea-4fd1-87de-955d2a24b5c8","Type":"ContainerStarted","Data":"38e0f2b0eafb9a854702be40f12827f0718b88ed8d27f42d67f45226332c01ab"}
Nov 04 02:13:38.800994 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:38.800922    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-mgqgj-m6j9p"]
Nov 04 02:13:38.805983 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:38.805967    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:38.909790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:38.909756    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 02:13:38.912944 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:38.912912    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-mgqgj-m6j9p"]
Nov 04 02:13:38.915871 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:38.915852    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-trusted-ca-bundle\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:38.915928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:38.915898    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-service-ca-bundle\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:38.916013 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:38.915998    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/6869b102-72dc-4949-b96d-30b1d207d320-archives-path\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:38.916116 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:38.916100    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dstrt\" (UniqueName: \"kubernetes.io/projected/6869b102-72dc-4949-b96d-30b1d207d320-kube-api-access-dstrt\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:38.959214 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:38.959186    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6wh47\""
Nov 04 02:13:39.010233 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.010202    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 02:13:39.017645 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.017620    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dstrt\" (UniqueName: \"kubernetes.io/projected/6869b102-72dc-4949-b96d-30b1d207d320-kube-api-access-dstrt\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:39.017747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.017677    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-trusted-ca-bundle\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:39.017747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.017716    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-service-ca-bundle\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:39.017843 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.017754    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/6869b102-72dc-4949-b96d-30b1d207d320-archives-path\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:39.018226 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.018202    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/6869b102-72dc-4949-b96d-30b1d207d320-archives-path\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:39.018430 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.018412    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-service-ca-bundle\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:39.018493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.018479    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-trusted-ca-bundle\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:39.079168 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.079096    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dstrt\" (UniqueName: \"kubernetes.io/projected/6869b102-72dc-4949-b96d-30b1d207d320-kube-api-access-dstrt\") pod \"periodic-gathering-mgqgj-m6j9p\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") " pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:39.095349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.095319    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7348-17efe5" event={"ID":"13f214d8-4eea-4fd1-87de-955d2a24b5c8","Type":"ContainerStarted","Data":"b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42"}
Nov 04 02:13:39.118175 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.118151    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:13:39.209298 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.209239    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7348-17efe5" podStartSLOduration=2.7576378889999997 podStartE2EDuration="5.209224956s" totalImagesPullingTime="2.451587067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:13:39.208193648 +0000 UTC m=+1081.013650280" watchObservedRunningTime="2026-11-04 02:13:39.209224956 +0000 UTC m=+1081.014681613"
Nov 04 02:13:39.277591 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.277553    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:39.494850 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:39.494828    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-mgqgj-m6j9p"]
Nov 04 02:13:39.496110 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:13:39.496088    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6869b102_72dc_4949_b96d_30b1d207d320.slice/crio-c8a6eb088fc8ff6b64d73bbe709c3b755e4efd456762ef2a1e0cd63b8673b03a WatchSource:0}: Error finding container c8a6eb088fc8ff6b64d73bbe709c3b755e4efd456762ef2a1e0cd63b8673b03a: Status 404 returned error can't find the container with id c8a6eb088fc8ff6b64d73bbe709c3b755e4efd456762ef2a1e0cd63b8673b03a
Nov 04 02:13:40.103465 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.103428    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-mgqgj-m6j9p" event={"ID":"6869b102-72dc-4949-b96d-30b1d207d320","Type":"ContainerStarted","Data":"3ba365cbd12354f12f07aedd851446990b587a9497db226460f662a5872cf41e"}
Nov 04 02:13:40.103942 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.103529    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-mgqgj-m6j9p" event={"ID":"6869b102-72dc-4949-b96d-30b1d207d320","Type":"ContainerStarted","Data":"c8a6eb088fc8ff6b64d73bbe709c3b755e4efd456762ef2a1e0cd63b8673b03a"}
Nov 04 02:13:40.230929 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.230876    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-mgqgj-m6j9p" podStartSLOduration=2.230862787 podStartE2EDuration="2.230862787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:13:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:13:40.229474952 +0000 UTC m=+1082.034931584" watchObservedRunningTime="2026-11-04 02:13:40.230862787 +0000 UTC m=+1082.036319418"
Nov 04 02:13:40.688092 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.688053    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7348-17efe5"]
Nov 04 02:13:40.693741 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.693712    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7348-17efe5"]
Nov 04 02:13:40.845367 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.845331    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7348-b78501"]
Nov 04 02:13:40.845918 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.845903    4512 cpu_manager.go:414] "containerMap: removing container" podUID="13f214d8-4eea-4fd1-87de-955d2a24b5c8" containerName="test-curl-7348-17efe5"
Nov 04 02:13:40.845972 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.845919    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13f214d8-4eea-4fd1-87de-955d2a24b5c8" containerName="test-curl-7348-17efe5"
Nov 04 02:13:40.846037 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.846028    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="13f214d8-4eea-4fd1-87de-955d2a24b5c8" containerName="test-curl-7348-17efe5"
Nov 04 02:13:40.849545 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.849524    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:40.902844 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.902814    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7348-b78501"]
Nov 04 02:13:40.937323 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:40.937286    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6qgb\" (UniqueName: \"kubernetes.io/projected/0fc12672-4989-467f-a212-48ce5961934e-kube-api-access-z6qgb\") pod \"test-curl-7348-b78501\" (UID: \"0fc12672-4989-467f-a212-48ce5961934e\") " pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:41.038325 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:41.038289    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6qgb\" (UniqueName: \"kubernetes.io/projected/0fc12672-4989-467f-a212-48ce5961934e-kube-api-access-z6qgb\") pod \"test-curl-7348-b78501\" (UID: \"0fc12672-4989-467f-a212-48ce5961934e\") " pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:41.099440 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:41.099403    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6qgb\" (UniqueName: \"kubernetes.io/projected/0fc12672-4989-467f-a212-48ce5961934e-kube-api-access-z6qgb\") pod \"test-curl-7348-b78501\" (UID: \"0fc12672-4989-467f-a212-48ce5961934e\") " pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:41.110158 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:41.110114    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7348-17efe5" podUID="13f214d8-4eea-4fd1-87de-955d2a24b5c8" containerName="test-curl-7348-17efe5" containerID="cri-o://b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42" gracePeriod=2
Nov 04 02:13:41.161403 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:41.161359    4512 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7348-17efe5\" is forbidden: User \"system:node:ip-10-0-142-191.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-191.us-west-1.compute.internal' and this object" podUID="13f214d8-4eea-4fd1-87de-955d2a24b5c8" pod="opendatahub/test-curl-7348-17efe5"
Nov 04 02:13:41.161646 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:41.161421    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:41.534900 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:41.534871    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7348-b78501"]
Nov 04 02:13:41.536069 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:13:41.536044    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fc12672_4989_467f_a212_48ce5961934e.slice/crio-5b19e0ea82f57b77a47e0bc7e1a69a6e6930c4c1a04011d93ee262568166c8c2 WatchSource:0}: Error finding container 5b19e0ea82f57b77a47e0bc7e1a69a6e6930c4c1a04011d93ee262568166c8c2: Status 404 returned error can't find the container with id 5b19e0ea82f57b77a47e0bc7e1a69a6e6930c4c1a04011d93ee262568166c8c2
Nov 04 02:13:42.118171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:42.118135    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7348-b78501" event={"ID":"0fc12672-4989-467f-a212-48ce5961934e","Type":"ContainerStarted","Data":"df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3"}
Nov 04 02:13:42.118171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:42.118179    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7348-b78501" event={"ID":"0fc12672-4989-467f-a212-48ce5961934e","Type":"ContainerStarted","Data":"5b19e0ea82f57b77a47e0bc7e1a69a6e6930c4c1a04011d93ee262568166c8c2"}
Nov 04 02:13:42.232222 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:42.232154    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7348-b78501" podStartSLOduration=2.232141914 podStartE2EDuration="2.232141914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:13:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:13:42.230810463 +0000 UTC m=+1084.036267098" watchObservedRunningTime="2026-11-04 02:13:42.232141914 +0000 UTC m=+1084.037598545"
Nov 04 02:13:42.281967 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:42.281923    4512 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7348-17efe5\" is forbidden: User \"system:node:ip-10-0-142-191.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-191.us-west-1.compute.internal' and this object" podUID="13f214d8-4eea-4fd1-87de-955d2a24b5c8" pod="opendatahub/test-curl-7348-17efe5"
Nov 04 02:13:43.354005 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:43.353977    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7348-17efe5"
Nov 04 02:13:43.464709 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:43.464606    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13f214d8-4eea-4fd1-87de-955d2a24b5c8-kube-api-access-jv2ft\" (UniqueName: \"kubernetes.io/projected/13f214d8-4eea-4fd1-87de-955d2a24b5c8-kube-api-access-jv2ft\") pod \"13f214d8-4eea-4fd1-87de-955d2a24b5c8\" (UID: \"13f214d8-4eea-4fd1-87de-955d2a24b5c8\") "
Nov 04 02:13:43.466841 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:43.466818    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13f214d8-4eea-4fd1-87de-955d2a24b5c8-kube-api-access-jv2ft" pod "13f214d8-4eea-4fd1-87de-955d2a24b5c8" (UID: "13f214d8-4eea-4fd1-87de-955d2a24b5c8"). InnerVolumeSpecName "kube-api-access-jv2ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:13:43.566261 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:43.566231    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv2ft\" (UniqueName: \"kubernetes.io/projected/13f214d8-4eea-4fd1-87de-955d2a24b5c8-kube-api-access-jv2ft\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:13:43.816256 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:43.816224    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7348-b78501"]
Nov 04 02:13:43.838021 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:43.837987    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7348-b78501"]
Nov 04 02:13:43.894733 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:43.894691    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:44.132711 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:44.132620    4512 generic.go:356] "Generic (PLEG): container finished" podID="13f214d8-4eea-4fd1-87de-955d2a24b5c8" containerID="b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42" exitCode=137
Nov 04 02:13:44.132711 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:44.132670    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7348-17efe5"
Nov 04 02:13:44.132909 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:44.132720    4512 scope.go:122] "RemoveContainer" containerID="b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42"
Nov 04 02:13:44.133015 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:44.132990    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7348-b78501" podUID="0fc12672-4989-467f-a212-48ce5961934e" containerName="test-curl-7348-b78501" containerID="cri-o://df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3" gracePeriod=2
Nov 04 02:13:44.147244 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:44.147218    4512 scope.go:122] "RemoveContainer" containerID="b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42"
Nov 04 02:13:44.147630 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:44.147608    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42\": container with ID starting with b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42 not found: ID does not exist" containerID="b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42"
Nov 04 02:13:44.147703 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:44.147639    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42"} err="failed to get container status \"b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42\": rpc error: code = NotFound desc = could not find container \"b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42\": container with ID starting with b724706242154b31b0c85213056b4459e370fd3848c147d1ccd1e6bc757eee42 not found: ID does not exist"
Nov 04 02:13:44.170884 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:44.170844    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:44.185099 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:44.184301    4512 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7348-b78501\" is forbidden: User \"system:node:ip-10-0-142-191.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-191.us-west-1.compute.internal' and this object" podUID="0fc12672-4989-467f-a212-48ce5961934e" pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:44.235559 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:44.235491    4512 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7348-b78501\" is forbidden: User \"system:node:ip-10-0-142-191.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-191.us-west-1.compute.internal' and this object" podUID="0fc12672-4989-467f-a212-48ce5961934e" pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:44.286570 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:44.286532    4512 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7348-17efe5\" is forbidden: User \"system:node:ip-10-0-142-191.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-191.us-west-1.compute.internal' and this object" podUID="13f214d8-4eea-4fd1-87de-955d2a24b5c8" pod="opendatahub/test-curl-7348-17efe5"
Nov 04 02:13:44.313285 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:44.313256    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13f214d8-4eea-4fd1-87de-955d2a24b5c8" path="/var/lib/kubelet/pods/13f214d8-4eea-4fd1-87de-955d2a24b5c8/volumes"
Nov 04 02:13:45.712576 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:45.712538    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:46.341010 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:46.340974    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:46.455719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:46.455696    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:46.494799 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:46.494769    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fc12672-4989-467f-a212-48ce5961934e-kube-api-access-z6qgb\" (UniqueName: \"kubernetes.io/projected/0fc12672-4989-467f-a212-48ce5961934e-kube-api-access-z6qgb\") pod \"0fc12672-4989-467f-a212-48ce5961934e\" (UID: \"0fc12672-4989-467f-a212-48ce5961934e\") "
Nov 04 02:13:46.496860 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:46.496837    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fc12672-4989-467f-a212-48ce5961934e-kube-api-access-z6qgb" pod "0fc12672-4989-467f-a212-48ce5961934e" (UID: "0fc12672-4989-467f-a212-48ce5961934e"). InnerVolumeSpecName "kube-api-access-z6qgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:13:46.596432 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:46.596401    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6qgb\" (UniqueName: \"kubernetes.io/projected/0fc12672-4989-467f-a212-48ce5961934e-kube-api-access-z6qgb\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:13:46.627004 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:46.626966    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:47.155171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:47.155135    4512 generic.go:356] "Generic (PLEG): container finished" podID="0fc12672-4989-467f-a212-48ce5961934e" containerID="df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3" exitCode=137
Nov 04 02:13:47.155605 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:47.155186    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:47.155605 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:47.155218    4512 scope.go:122] "RemoveContainer" containerID="df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3"
Nov 04 02:13:47.164830 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:47.164812    4512 scope.go:122] "RemoveContainer" containerID="df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3"
Nov 04 02:13:47.165094 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:47.165077    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3\": container with ID starting with df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3 not found: ID does not exist" containerID="df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3"
Nov 04 02:13:47.165131 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:47.165104    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3"} err="failed to get container status \"df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3\": rpc error: code = NotFound desc = could not find container \"df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3\": container with ID starting with df9c8c02f9b0a7f3f6306ce76e02e05ac9d1fb4554e4cb526ccafb01b08fa4c3 not found: ID does not exist"
Nov 04 02:13:47.221922 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:47.221890    4512 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7348-b78501\" is forbidden: User \"system:node:ip-10-0-142-191.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-191.us-west-1.compute.internal' and this object" podUID="0fc12672-4989-467f-a212-48ce5961934e" pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:48.314407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:48.314368    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fc12672-4989-467f-a212-48ce5961934e" path="/var/lib/kubelet/pods/0fc12672-4989-467f-a212-48ce5961934e/volumes"
Nov 04 02:13:48.362359 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:48.362316    4512 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7348-b78501\" is forbidden: User \"system:node:ip-10-0-142-191.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-191.us-west-1.compute.internal' and this object" podUID="0fc12672-4989-467f-a212-48ce5961934e" pod="opendatahub/test-curl-7348-b78501"
Nov 04 02:13:49.172335 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:49.172295    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:50.294277 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:50.294238    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:50.549915 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:50.549835    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:53.554488 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.554446    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:53.763487 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.763444    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:53.913171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.913090    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"]
Nov 04 02:13:53.913691 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.913668    4512 cpu_manager.go:414] "containerMap: removing container" podUID="0fc12672-4989-467f-a212-48ce5961934e" containerName="test-curl-7348-b78501"
Nov 04 02:13:53.913737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.913692    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fc12672-4989-467f-a212-48ce5961934e" containerName="test-curl-7348-b78501"
Nov 04 02:13:53.913845 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.913835    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fc12672-4989-467f-a212-48ce5961934e" containerName="test-curl-7348-b78501"
Nov 04 02:13:53.918719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.918701    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:13:53.971145 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.971113    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-a8ef09\""
Nov 04 02:13:53.976119 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.976078    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"]
Nov 04 02:13:53.980709 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.980679    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adad2e2b-654d-459e-a642-308912801cb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll\" (UID: \"adad2e2b-654d-459e-a642-308912801cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:13:53.981306 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:53.980742    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88wv2\" (UniqueName: \"kubernetes.io/projected/adad2e2b-654d-459e-a642-308912801cb2-kube-api-access-88wv2\") pod \"payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll\" (UID: \"adad2e2b-654d-459e-a642-308912801cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:13:54.025864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.025835    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-a8ef09-dockercfg-thqm9\""
Nov 04 02:13:54.063536 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.063472    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"]
Nov 04 02:13:54.067864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.067840    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:13:54.081743 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.081712    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adad2e2b-654d-459e-a642-308912801cb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll\" (UID: \"adad2e2b-654d-459e-a642-308912801cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:13:54.081906 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.081775    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88wv2\" (UniqueName: \"kubernetes.io/projected/adad2e2b-654d-459e-a642-308912801cb2-kube-api-access-88wv2\") pod \"payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll\" (UID: \"adad2e2b-654d-459e-a642-308912801cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:13:54.082432 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.082410    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adad2e2b-654d-459e-a642-308912801cb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll\" (UID: \"adad2e2b-654d-459e-a642-308912801cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:13:54.126119 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.126084    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"]
Nov 04 02:13:54.143144 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.143107    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88wv2\" (UniqueName: \"kubernetes.io/projected/adad2e2b-654d-459e-a642-308912801cb2-kube-api-access-88wv2\") pod \"payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll\" (UID: \"adad2e2b-654d-459e-a642-308912801cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:13:54.182972 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.182865    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54h6d\" (UniqueName: \"kubernetes.io/projected/902d38f5-d5c2-429c-9bb2-d39e34660dad-kube-api-access-54h6d\") pod \"payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6\" (UID: \"902d38f5-d5c2-429c-9bb2-d39e34660dad\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:13:54.183140 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.183031    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/902d38f5-d5c2-429c-9bb2-d39e34660dad-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6\" (UID: \"902d38f5-d5c2-429c-9bb2-d39e34660dad\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:13:54.235555 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.235500    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:13:54.284341 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.284296    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54h6d\" (UniqueName: \"kubernetes.io/projected/902d38f5-d5c2-429c-9bb2-d39e34660dad-kube-api-access-54h6d\") pod \"payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6\" (UID: \"902d38f5-d5c2-429c-9bb2-d39e34660dad\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:13:54.284536 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.284435    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/902d38f5-d5c2-429c-9bb2-d39e34660dad-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6\" (UID: \"902d38f5-d5c2-429c-9bb2-d39e34660dad\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:13:54.285391 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.285366    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/902d38f5-d5c2-429c-9bb2-d39e34660dad-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6\" (UID: \"902d38f5-d5c2-429c-9bb2-d39e34660dad\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:13:54.345799 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.345727    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54h6d\" (UniqueName: \"kubernetes.io/projected/902d38f5-d5c2-429c-9bb2-d39e34660dad-kube-api-access-54h6d\") pod \"payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6\" (UID: \"902d38f5-d5c2-429c-9bb2-d39e34660dad\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:13:54.382321 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.382266    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:13:54.484001 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.480839    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:54.618563 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.618532    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"]
Nov 04 02:13:54.620031 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:13:54.620004    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadad2e2b_654d_459e_a642_308912801cb2.slice/crio-a9101c1fbdedb9e996bf9030c6ce691ad79f83f5807c1cb4421ab60ef78476b9 WatchSource:0}: Error finding container a9101c1fbdedb9e996bf9030c6ce691ad79f83f5807c1cb4421ab60ef78476b9: Status 404 returned error can't find the container with id a9101c1fbdedb9e996bf9030c6ce691ad79f83f5807c1cb4421ab60ef78476b9
Nov 04 02:13:54.800257 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:54.800219    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"]
Nov 04 02:13:54.800605 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:13:54.800573    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902d38f5_d5c2_429c_9bb2_d39e34660dad.slice/crio-ed7ae5c87b63537a26eac180b9f35c5c24f891424e6ae0d1edcb10883d87df8a WatchSource:0}: Error finding container ed7ae5c87b63537a26eac180b9f35c5c24f891424e6ae0d1edcb10883d87df8a: Status 404 returned error can't find the container with id ed7ae5c87b63537a26eac180b9f35c5c24f891424e6ae0d1edcb10883d87df8a
Nov 04 02:13:55.219560 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:55.219443    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6" event={"ID":"902d38f5-d5c2-429c-9bb2-d39e34660dad","Type":"ContainerStarted","Data":"2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791"}
Nov 04 02:13:55.219560 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:55.219554    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6" event={"ID":"902d38f5-d5c2-429c-9bb2-d39e34660dad","Type":"ContainerStarted","Data":"ed7ae5c87b63537a26eac180b9f35c5c24f891424e6ae0d1edcb10883d87df8a"}
Nov 04 02:13:55.219811 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:55.219793    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:13:55.221173 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:55.221147    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll" event={"ID":"adad2e2b-654d-459e-a642-308912801cb2","Type":"ContainerStarted","Data":"a3a3decff7c4976df1b178c14e20431df174d4054467d075481a74f60ddf0456"}
Nov 04 02:13:55.221301 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:55.221188    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll" event={"ID":"adad2e2b-654d-459e-a642-308912801cb2","Type":"ContainerStarted","Data":"a9101c1fbdedb9e996bf9030c6ce691ad79f83f5807c1cb4421ab60ef78476b9"}
Nov 04 02:13:55.221301 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:55.221215    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:13:55.338252 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:55.338183    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6" podStartSLOduration=1.33815821 podStartE2EDuration="1.33815821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:13:55.335498388 +0000 UTC m=+1097.140955020" watchObservedRunningTime="2026-11-04 02:13:55.33815821 +0000 UTC m=+1097.143614843"
Nov 04 02:13:55.456373 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:55.456306    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll" podStartSLOduration=2.456289671 podStartE2EDuration="2.456289671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:13:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:13:55.451069602 +0000 UTC m=+1097.256526234" watchObservedRunningTime="2026-11-04 02:13:55.456289671 +0000 UTC m=+1097.261746337"
Nov 04 02:13:56.778700 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:13:56.778667    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:13:57.536366 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:57.536314    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:59.490837 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:13:59.490798    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:04.386543 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:04.385636    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:06.232886 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:06.232857    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:14:06.233349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:06.232928    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:14:09.307763 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:09.307716    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:14.286720 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:14.286683    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:16.575124 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:16.575078    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:19.135604 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:19.135429    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:19.332918 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:19.332872    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:19.528684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:19.528645    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:24.185551 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:24.185487    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:26.780191 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:14:26.780153    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:14:29.069499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:29.069457    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:30.491396 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:30.491351    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:33.232902 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.232868    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"]
Nov 04 02:14:33.237196 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.237177    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.296492 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.296456    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"]
Nov 04 02:14:33.341264 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.341209    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"openshift-service-ca.crt\""
Nov 04 02:14:33.341264 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.341231    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"default-dockercfg-cz6zr\""
Nov 04 02:14:33.341264 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.341239    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"distinct-2-w0-24bea9-kserve-self-signed-certs\""
Nov 04 02:14:33.341264 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.341254    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"kube-root-ca.crt\""
Nov 04 02:14:33.364062 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.364024    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-model-cache\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.364247 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.364093    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b44b714-aa08-4392-b4f4-f7fafc232d42-tls-certs\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.364247 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.364201    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-dshm\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.364247 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.364235    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-tmp-dir\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.364404 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.364290    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-home\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.364404 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.364394    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssk24\" (UniqueName: \"kubernetes.io/projected/5b44b714-aa08-4392-b4f4-f7fafc232d42-kube-api-access-ssk24\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.389756 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.389714    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:33.465830 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.465793    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-dshm\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.466008 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.465857    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-tmp-dir\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.466008 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.465877    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-home\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.466008 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.465911    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssk24\" (UniqueName: \"kubernetes.io/projected/5b44b714-aa08-4392-b4f4-f7fafc232d42-kube-api-access-ssk24\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.466237 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.466015    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-model-cache\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.466237 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.466069    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b44b714-aa08-4392-b4f4-f7fafc232d42-tls-certs\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.466343 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.466309    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-home\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.466343 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.466323    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-tmp-dir\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.466420 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.466383    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-model-cache\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.468171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.468148    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-dshm\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.468480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.468463    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b44b714-aa08-4392-b4f4-f7fafc232d42-tls-certs\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.526825 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.526753    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssk24\" (UniqueName: \"kubernetes.io/projected/5b44b714-aa08-4392-b4f4-f7fafc232d42-kube-api-access-ssk24\") pod \"distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") " pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.552296 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.552263    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:33.941262 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:14:33.941231    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b44b714_aa08_4392_b4f4_f7fafc232d42.slice/crio-34305e8041931754401ce4e472d1fa1f283cca556a30b96c7ab1a43b01659edc WatchSource:0}: Error finding container 34305e8041931754401ce4e472d1fa1f283cca556a30b96c7ab1a43b01659edc: Status 404 returned error can't find the container with id 34305e8041931754401ce4e472d1fa1f283cca556a30b96c7ab1a43b01659edc
Nov 04 02:14:33.941782 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.941740    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"]
Nov 04 02:14:33.995027 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:33.994983    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:34.526428 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:34.526381    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j" event={"ID":"5b44b714-aa08-4392-b4f4-f7fafc232d42","Type":"ContainerStarted","Data":"f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252"}
Nov 04 02:14:34.526428 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:34.526427    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j" event={"ID":"5b44b714-aa08-4392-b4f4-f7fafc232d42","Type":"ContainerStarted","Data":"34305e8041931754401ce4e472d1fa1f283cca556a30b96c7ab1a43b01659edc"}
Nov 04 02:14:34.552795 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:34.552762    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:34.643294 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:34.643138    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j" podStartSLOduration=1.643120267 podStartE2EDuration="1.643120267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:34.642813886 +0000 UTC m=+1136.448270517" watchObservedRunningTime="2026-11-04 02:14:34.643120267 +0000 UTC m=+1136.448576898"
Nov 04 02:14:38.890155 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:38.890057    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:43.553047 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:43.553009    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:43.566406 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:43.566384    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:43.617633 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:43.617598    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:14:43.872850 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:43.872755    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:48.771303 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:48.771259    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:53.683014 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:53.682975    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:56.778920 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:14:56.778884    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:14:58.586494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:58.586449    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:58.838608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:14:58.838477    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:00.176753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.176714    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"]
Nov 04 02:15:00.181959 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.181933    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"]
Nov 04 02:15:00.182094 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.182045    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"
Nov 04 02:15:00.186106 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.186084    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"]
Nov 04 02:15:00.186231 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.186211    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"
Nov 04 02:15:00.189744 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.189723    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"
Nov 04 02:15:00.225696 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.225667    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"]
Nov 04 02:15:00.229360 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.229342    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"]
Nov 04 02:15:00.229467 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.229451    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"
Nov 04 02:15:00.232980 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.232961    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"
Nov 04 02:15:00.234185 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.234162    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:15:00.265003 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.264977    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tm5l\" (UniqueName: \"kubernetes.io/projected/23c08a30-a814-42a4-bee0-bb5a7e39aa01-kube-api-access-5tm5l\") pod \"maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46\" (UID: \"23c08a30-a814-42a4-bee0-bb5a7e39aa01\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"
Nov 04 02:15:00.265148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.265018    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b946ba4-63d4-4019-8c5c-7e7f324720e5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv\" (UID: \"4b946ba4-63d4-4019-8c5c-7e7f324720e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"
Nov 04 02:15:00.265148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.265037    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4sz7\" (UniqueName: \"kubernetes.io/projected/4b946ba4-63d4-4019-8c5c-7e7f324720e5-kube-api-access-j4sz7\") pod \"maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv\" (UID: \"4b946ba4-63d4-4019-8c5c-7e7f324720e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"
Nov 04 02:15:00.265271 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.265209    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd9fb62-802b-4fdc-aafd-4470759d9796-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749\" (UID: \"1cd9fb62-802b-4fdc-aafd-4470759d9796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"
Nov 04 02:15:00.265346 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.265328    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdprf\" (UniqueName: \"kubernetes.io/projected/1cd9fb62-802b-4fdc-aafd-4470759d9796-kube-api-access-tdprf\") pod \"maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749\" (UID: \"1cd9fb62-802b-4fdc-aafd-4470759d9796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"
Nov 04 02:15:00.265405 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.265392    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c08a30-a814-42a4-bee0-bb5a7e39aa01-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46\" (UID: \"23c08a30-a814-42a4-bee0-bb5a7e39aa01\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"
Nov 04 02:15:00.274349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.274321    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"]
Nov 04 02:15:00.279563 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.279539    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"]
Nov 04 02:15:00.284028 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.284000    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"]
Nov 04 02:15:00.323236 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.323193    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"]
Nov 04 02:15:00.326524 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.326481    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"]
Nov 04 02:15:00.366641 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.366604    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90ae9297-d8ef-454f-9c98-e8adc32e3c93-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895975-fw2zn\" (UID: \"90ae9297-d8ef-454f-9c98-e8adc32e3c93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"
Nov 04 02:15:00.366806 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.366680    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdprf\" (UniqueName: \"kubernetes.io/projected/1cd9fb62-802b-4fdc-aafd-4470759d9796-kube-api-access-tdprf\") pod \"maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749\" (UID: \"1cd9fb62-802b-4fdc-aafd-4470759d9796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"
Nov 04 02:15:00.366806 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.366721    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n55c2\" (UniqueName: \"kubernetes.io/projected/764e8d02-1f67-406d-a4b3-fe07edec30e7-kube-api-access-n55c2\") pod \"maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8\" (UID: \"764e8d02-1f67-406d-a4b3-fe07edec30e7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"
Nov 04 02:15:00.366929 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.366852    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c08a30-a814-42a4-bee0-bb5a7e39aa01-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46\" (UID: \"23c08a30-a814-42a4-bee0-bb5a7e39aa01\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"
Nov 04 02:15:00.366929 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.366880    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76h8j\" (UniqueName: \"kubernetes.io/projected/90ae9297-d8ef-454f-9c98-e8adc32e3c93-kube-api-access-76h8j\") pod \"maas-api-key-cleanup-29895975-fw2zn\" (UID: \"90ae9297-d8ef-454f-9c98-e8adc32e3c93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"
Nov 04 02:15:00.367018 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.366950    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tm5l\" (UniqueName: \"kubernetes.io/projected/23c08a30-a814-42a4-bee0-bb5a7e39aa01-kube-api-access-5tm5l\") pod \"maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46\" (UID: \"23c08a30-a814-42a4-bee0-bb5a7e39aa01\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"
Nov 04 02:15:00.367065 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.367046    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b946ba4-63d4-4019-8c5c-7e7f324720e5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv\" (UID: \"4b946ba4-63d4-4019-8c5c-7e7f324720e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"
Nov 04 02:15:00.367108 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.367076    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/764e8d02-1f67-406d-a4b3-fe07edec30e7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8\" (UID: \"764e8d02-1f67-406d-a4b3-fe07edec30e7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"
Nov 04 02:15:00.367157 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.367107    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4sz7\" (UniqueName: \"kubernetes.io/projected/4b946ba4-63d4-4019-8c5c-7e7f324720e5-kube-api-access-j4sz7\") pod \"maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv\" (UID: \"4b946ba4-63d4-4019-8c5c-7e7f324720e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"
Nov 04 02:15:00.367210 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.367195    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd9fb62-802b-4fdc-aafd-4470759d9796-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749\" (UID: \"1cd9fb62-802b-4fdc-aafd-4470759d9796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"
Nov 04 02:15:00.367756 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.367723    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c08a30-a814-42a4-bee0-bb5a7e39aa01-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46\" (UID: \"23c08a30-a814-42a4-bee0-bb5a7e39aa01\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"
Nov 04 02:15:00.367881 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.367866    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd9fb62-802b-4fdc-aafd-4470759d9796-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749\" (UID: \"1cd9fb62-802b-4fdc-aafd-4470759d9796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"
Nov 04 02:15:00.367919 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.367889    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b946ba4-63d4-4019-8c5c-7e7f324720e5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv\" (UID: \"4b946ba4-63d4-4019-8c5c-7e7f324720e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"
Nov 04 02:15:00.428951 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.428868    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tm5l\" (UniqueName: \"kubernetes.io/projected/23c08a30-a814-42a4-bee0-bb5a7e39aa01-kube-api-access-5tm5l\") pod \"maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46\" (UID: \"23c08a30-a814-42a4-bee0-bb5a7e39aa01\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"
Nov 04 02:15:00.428951 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.428889    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4sz7\" (UniqueName: \"kubernetes.io/projected/4b946ba4-63d4-4019-8c5c-7e7f324720e5-kube-api-access-j4sz7\") pod \"maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv\" (UID: \"4b946ba4-63d4-4019-8c5c-7e7f324720e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"
Nov 04 02:15:00.428951 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.428948    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdprf\" (UniqueName: \"kubernetes.io/projected/1cd9fb62-802b-4fdc-aafd-4470759d9796-kube-api-access-tdprf\") pod \"maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749\" (UID: \"1cd9fb62-802b-4fdc-aafd-4470759d9796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"
Nov 04 02:15:00.469017 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.468982    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76h8j\" (UniqueName: \"kubernetes.io/projected/90ae9297-d8ef-454f-9c98-e8adc32e3c93-kube-api-access-76h8j\") pod \"maas-api-key-cleanup-29895975-fw2zn\" (UID: \"90ae9297-d8ef-454f-9c98-e8adc32e3c93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"
Nov 04 02:15:00.469216 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.469131    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/764e8d02-1f67-406d-a4b3-fe07edec30e7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8\" (UID: \"764e8d02-1f67-406d-a4b3-fe07edec30e7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"
Nov 04 02:15:00.469284 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.469258    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90ae9297-d8ef-454f-9c98-e8adc32e3c93-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895975-fw2zn\" (UID: \"90ae9297-d8ef-454f-9c98-e8adc32e3c93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"
Nov 04 02:15:00.469339 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.469326    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n55c2\" (UniqueName: \"kubernetes.io/projected/764e8d02-1f67-406d-a4b3-fe07edec30e7-kube-api-access-n55c2\") pod \"maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8\" (UID: \"764e8d02-1f67-406d-a4b3-fe07edec30e7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"
Nov 04 02:15:00.469915 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.469893    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/764e8d02-1f67-406d-a4b3-fe07edec30e7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8\" (UID: \"764e8d02-1f67-406d-a4b3-fe07edec30e7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"
Nov 04 02:15:00.470083 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.470057    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90ae9297-d8ef-454f-9c98-e8adc32e3c93-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895975-fw2zn\" (UID: \"90ae9297-d8ef-454f-9c98-e8adc32e3c93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"
Nov 04 02:15:00.498769 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.498739    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"
Nov 04 02:15:00.508012 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.507981    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"
Nov 04 02:15:00.514049 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.514028    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"
Nov 04 02:15:00.530431 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.530401    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76h8j\" (UniqueName: \"kubernetes.io/projected/90ae9297-d8ef-454f-9c98-e8adc32e3c93-kube-api-access-76h8j\") pod \"maas-api-key-cleanup-29895975-fw2zn\" (UID: \"90ae9297-d8ef-454f-9c98-e8adc32e3c93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"
Nov 04 02:15:00.530989 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.530966    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n55c2\" (UniqueName: \"kubernetes.io/projected/764e8d02-1f67-406d-a4b3-fe07edec30e7-kube-api-access-n55c2\") pod \"maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8\" (UID: \"764e8d02-1f67-406d-a4b3-fe07edec30e7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"
Nov 04 02:15:00.544460 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.544433    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"
Nov 04 02:15:00.550785 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.550672    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"
Nov 04 02:15:00.897478 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.897448    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"]
Nov 04 02:15:00.898382 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:15:00.898347    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c08a30_a814_42a4_bee0_bb5a7e39aa01.slice/crio-d04e06cb2b55e245dade66ecc2616dbe2ab000d8e5a7afb3ca612908aa9587fc WatchSource:0}: Error finding container d04e06cb2b55e245dade66ecc2616dbe2ab000d8e5a7afb3ca612908aa9587fc: Status 404 returned error can't find the container with id d04e06cb2b55e245dade66ecc2616dbe2ab000d8e5a7afb3ca612908aa9587fc
Nov 04 02:15:00.945555 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.945531    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"]
Nov 04 02:15:00.945649 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:15:00.945500    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd9fb62_802b_4fdc_aafd_4470759d9796.slice/crio-2f8f6a4e9267e5b5c120b25ab753b902a7a1ede2bd3bb474abfde8e91b9a4210 WatchSource:0}: Error finding container 2f8f6a4e9267e5b5c120b25ab753b902a7a1ede2bd3bb474abfde8e91b9a4210: Status 404 returned error can't find the container with id 2f8f6a4e9267e5b5c120b25ab753b902a7a1ede2bd3bb474abfde8e91b9a4210
Nov 04 02:15:00.946537 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:15:00.946494    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b946ba4_63d4_4019_8c5c_7e7f324720e5.slice/crio-e59bfea7fef2de35bc40f2c359c3e58cb9b935675436a407eb1272d0ae103f52 WatchSource:0}: Error finding container e59bfea7fef2de35bc40f2c359c3e58cb9b935675436a407eb1272d0ae103f52: Status 404 returned error can't find the container with id e59bfea7fef2de35bc40f2c359c3e58cb9b935675436a407eb1272d0ae103f52
Nov 04 02:15:00.949043 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.948889    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"]
Nov 04 02:15:00.966544 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:00.966519    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"]
Nov 04 02:15:00.967715 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:15:00.967690    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ae9297_d8ef_454f_9c98_e8adc32e3c93.slice/crio-df4b911230895ddbe23a8eef977c0a92b258138ee434239c14a80d7a2013070e WatchSource:0}: Error finding container df4b911230895ddbe23a8eef977c0a92b258138ee434239c14a80d7a2013070e: Status 404 returned error can't find the container with id df4b911230895ddbe23a8eef977c0a92b258138ee434239c14a80d7a2013070e
Nov 04 02:15:00.997604 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:15:00.997580    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod764e8d02_1f67_406d_a4b3_fe07edec30e7.slice/crio-afa0d4cc7c021bf2a4e8fc6ec8991fee53cacbe0c35d031ddd376e21e5cf49d1 WatchSource:0}: Error finding container afa0d4cc7c021bf2a4e8fc6ec8991fee53cacbe0c35d031ddd376e21e5cf49d1: Status 404 returned error can't find the container with id afa0d4cc7c021bf2a4e8fc6ec8991fee53cacbe0c35d031ddd376e21e5cf49d1
Nov 04 02:15:01.000498 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:01.000421    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"]
Nov 04 02:15:01.619670 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:01.619632    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:01.733815 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:01.733776    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8" event={"ID":"764e8d02-1f67-406d-a4b3-fe07edec30e7","Type":"ContainerStarted","Data":"afa0d4cc7c021bf2a4e8fc6ec8991fee53cacbe0c35d031ddd376e21e5cf49d1"}
Nov 04 02:15:01.735119 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:01.735090    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn" event={"ID":"90ae9297-d8ef-454f-9c98-e8adc32e3c93","Type":"ContainerStarted","Data":"df4b911230895ddbe23a8eef977c0a92b258138ee434239c14a80d7a2013070e"}
Nov 04 02:15:01.737030 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:01.737002    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" event={"ID":"1cd9fb62-802b-4fdc-aafd-4470759d9796","Type":"ContainerStarted","Data":"2f8f6a4e9267e5b5c120b25ab753b902a7a1ede2bd3bb474abfde8e91b9a4210"}
Nov 04 02:15:01.738257 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:01.738235    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" event={"ID":"23c08a30-a814-42a4-bee0-bb5a7e39aa01","Type":"ContainerStarted","Data":"d04e06cb2b55e245dade66ecc2616dbe2ab000d8e5a7afb3ca612908aa9587fc"}
Nov 04 02:15:01.739349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:01.739329    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" event={"ID":"4b946ba4-63d4-4019-8c5c-7e7f324720e5","Type":"ContainerStarted","Data":"e59bfea7fef2de35bc40f2c359c3e58cb9b935675436a407eb1272d0ae103f52"}
Nov 04 02:15:03.006870 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:03.006838    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:03.256702 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:03.256647    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:03.802236 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:03.799164    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:04.007996 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:04.007962    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:04.214880 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:04.214786    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:04.430768 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:04.430732    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:06.792961 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:06.792924    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" event={"ID":"1cd9fb62-802b-4fdc-aafd-4470759d9796","Type":"ContainerStarted","Data":"2f8e6330a1575268869c2e1061e5b55b36ab9cdcb11625587e8d1238dd43f947"}
Nov 04 02:15:06.795012 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:06.794933    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" event={"ID":"23c08a30-a814-42a4-bee0-bb5a7e39aa01","Type":"ContainerStarted","Data":"35bbea5cae650406ac0ecb9363a41b93379d89fd2f0d2b32e6c6a31771cf998f"}
Nov 04 02:15:06.910103 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:06.910048    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" podStartSLOduration=1.458491966 podStartE2EDuration="6.910031932s" totalImagesPullingTime="5.451539966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:06.908134025 +0000 UTC m=+1168.713590649" watchObservedRunningTime="2026-11-04 02:15:06.910031932 +0000 UTC m=+1168.715488563"
Nov 04 02:15:07.805673 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.805646    4512 generic.go:356] "Generic (PLEG): container finished" podID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerID="ece65f58d3e6b09709e387fc34b7522f5c45cb34f28d7e74066f48dc3c23687e" exitCode=60
Nov 04 02:15:07.806032 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.805731    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8" event={"ID":"764e8d02-1f67-406d-a4b3-fe07edec30e7","Type":"ContainerDied","Data":"ece65f58d3e6b09709e387fc34b7522f5c45cb34f28d7e74066f48dc3c23687e"}
Nov 04 02:15:07.806032 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.806017    4512 scope.go:122] "RemoveContainer" containerID="ece65f58d3e6b09709e387fc34b7522f5c45cb34f28d7e74066f48dc3c23687e"
Nov 04 02:15:07.807418 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.807399    4512 generic.go:356] "Generic (PLEG): container finished" podID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerID="7f9a742c92b69121ac5eee8529e9021cea83753e89894195f2c72c5a651f719b" exitCode=60
Nov 04 02:15:07.807527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.807443    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn" event={"ID":"90ae9297-d8ef-454f-9c98-e8adc32e3c93","Type":"ContainerDied","Data":"7f9a742c92b69121ac5eee8529e9021cea83753e89894195f2c72c5a651f719b"}
Nov 04 02:15:07.807775 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.807758    4512 scope.go:122] "RemoveContainer" containerID="7f9a742c92b69121ac5eee8529e9021cea83753e89894195f2c72c5a651f719b"
Nov 04 02:15:07.809191 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.809171    4512 generic.go:356] "Generic (PLEG): container finished" podID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerID="2f8e6330a1575268869c2e1061e5b55b36ab9cdcb11625587e8d1238dd43f947" exitCode=60
Nov 04 02:15:07.810662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.809555    4512 scope.go:122] "RemoveContainer" containerID="2f8e6330a1575268869c2e1061e5b55b36ab9cdcb11625587e8d1238dd43f947"
Nov 04 02:15:07.810662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.809554    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" event={"ID":"1cd9fb62-802b-4fdc-aafd-4470759d9796","Type":"ContainerDied","Data":"2f8e6330a1575268869c2e1061e5b55b36ab9cdcb11625587e8d1238dd43f947"}
Nov 04 02:15:07.810662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.810368    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:07.811722 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.811701    4512 generic.go:356] "Generic (PLEG): container finished" podID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerID="35bbea5cae650406ac0ecb9363a41b93379d89fd2f0d2b32e6c6a31771cf998f" exitCode=60
Nov 04 02:15:07.811818 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.811791    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" event={"ID":"23c08a30-a814-42a4-bee0-bb5a7e39aa01","Type":"ContainerDied","Data":"35bbea5cae650406ac0ecb9363a41b93379d89fd2f0d2b32e6c6a31771cf998f"}
Nov 04 02:15:07.812127 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.811957    4512 scope.go:122] "RemoveContainer" containerID="35bbea5cae650406ac0ecb9363a41b93379d89fd2f0d2b32e6c6a31771cf998f"
Nov 04 02:15:07.813441 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.813423    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" event={"ID":"4b946ba4-63d4-4019-8c5c-7e7f324720e5","Type":"ContainerStarted","Data":"d1fe5af7f0acc3fd3e84ff19947c27b67807ff582503daf8cdf20d084163bf4f"}
Nov 04 02:15:07.813587 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.813573    4512 scope.go:122] "RemoveContainer" containerID="d1fe5af7f0acc3fd3e84ff19947c27b67807ff582503daf8cdf20d084163bf4f"
Nov 04 02:15:07.927667 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:07.927613    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" podStartSLOduration=2.6837414490000002 podStartE2EDuration="7.927597903s" totalImagesPullingTime="5.243856454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:07.025336273 +0000 UTC m=+1168.830792904" watchObservedRunningTime="2026-11-04 02:15:07.927597903 +0000 UTC m=+1169.733054534"
Nov 04 02:15:08.066874 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.066836    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:08.821589 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.821557    4512 generic.go:356] "Generic (PLEG): container finished" podID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerID="b423ca406b7a752816613e107a95d3b162f5acfebdc90f4b432caedf3831e4bb" exitCode=60
Nov 04 02:15:08.821938 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.821637    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" event={"ID":"23c08a30-a814-42a4-bee0-bb5a7e39aa01","Type":"ContainerDied","Data":"b423ca406b7a752816613e107a95d3b162f5acfebdc90f4b432caedf3831e4bb"}
Nov 04 02:15:08.821938 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.821721    4512 scope.go:122] "RemoveContainer" containerID="35bbea5cae650406ac0ecb9363a41b93379d89fd2f0d2b32e6c6a31771cf998f"
Nov 04 02:15:08.822055 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.822009    4512 scope.go:122] "RemoveContainer" containerID="b423ca406b7a752816613e107a95d3b162f5acfebdc90f4b432caedf3831e4bb"
Nov 04 02:15:08.822354 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:08.822329    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46_odh-ai-gateway-infra(23c08a30-a814-42a4-bee0-bb5a7e39aa01)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01"
Nov 04 02:15:08.823739 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.823720    4512 generic.go:356] "Generic (PLEG): container finished" podID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerID="d1fe5af7f0acc3fd3e84ff19947c27b67807ff582503daf8cdf20d084163bf4f" exitCode=60
Nov 04 02:15:08.823827 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.823804    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" event={"ID":"4b946ba4-63d4-4019-8c5c-7e7f324720e5","Type":"ContainerDied","Data":"d1fe5af7f0acc3fd3e84ff19947c27b67807ff582503daf8cdf20d084163bf4f"}
Nov 04 02:15:08.823890 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.823854    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" event={"ID":"4b946ba4-63d4-4019-8c5c-7e7f324720e5","Type":"ContainerStarted","Data":"3036f4d8384986b69bdfe47d7b66f05d4638dd7c892f521bdece50baf2569c14"}
Nov 04 02:15:08.825465 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.825442    4512 generic.go:356] "Generic (PLEG): container finished" podID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerID="09c3e7c81dd1d9d3c0f0824738289f9a3b3256fbab0f7d3736ed2406fb716468" exitCode=60
Nov 04 02:15:08.825572 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.825471    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8" event={"ID":"764e8d02-1f67-406d-a4b3-fe07edec30e7","Type":"ContainerDied","Data":"09c3e7c81dd1d9d3c0f0824738289f9a3b3256fbab0f7d3736ed2406fb716468"}
Nov 04 02:15:08.825782 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.825763    4512 scope.go:122] "RemoveContainer" containerID="09c3e7c81dd1d9d3c0f0824738289f9a3b3256fbab0f7d3736ed2406fb716468"
Nov 04 02:15:08.826054 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:08.826029    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8_odh-ai-gateway-infra(764e8d02-1f67-406d-a4b3-fe07edec30e7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7"
Nov 04 02:15:08.827279 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.827257    4512 generic.go:356] "Generic (PLEG): container finished" podID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerID="72ef14b4edf480c40b9bd3538383e34115f70c7e3cade09a74744f6dd21509db" exitCode=60
Nov 04 02:15:08.827375 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.827319    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn" event={"ID":"90ae9297-d8ef-454f-9c98-e8adc32e3c93","Type":"ContainerDied","Data":"72ef14b4edf480c40b9bd3538383e34115f70c7e3cade09a74744f6dd21509db"}
Nov 04 02:15:08.827578 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.827561    4512 scope.go:122] "RemoveContainer" containerID="72ef14b4edf480c40b9bd3538383e34115f70c7e3cade09a74744f6dd21509db"
Nov 04 02:15:08.827832 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:08.827799    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895975-fw2zn_odh-ai-gateway-infra(90ae9297-d8ef-454f-9c98-e8adc32e3c93)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93"
Nov 04 02:15:08.829113 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.829094    4512 generic.go:356] "Generic (PLEG): container finished" podID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerID="b24e7dc3b6fa0604c2c32d7f3a5bc5dbed418507019dfb1fa5ca744d4fbdb37a" exitCode=60
Nov 04 02:15:08.829198 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.829187    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" event={"ID":"1cd9fb62-802b-4fdc-aafd-4470759d9796","Type":"ContainerDied","Data":"b24e7dc3b6fa0604c2c32d7f3a5bc5dbed418507019dfb1fa5ca744d4fbdb37a"}
Nov 04 02:15:08.829303 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.829289    4512 scope.go:122] "RemoveContainer" containerID="b24e7dc3b6fa0604c2c32d7f3a5bc5dbed418507019dfb1fa5ca744d4fbdb37a"
Nov 04 02:15:08.829482 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:08.829467    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749_odh-ai-gateway-infra(1cd9fb62-802b-4fdc-aafd-4470759d9796)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796"
Nov 04 02:15:08.960242 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:08.960204    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:09.064706 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.064680    4512 scope.go:122] "RemoveContainer" containerID="ece65f58d3e6b09709e387fc34b7522f5c45cb34f28d7e74066f48dc3c23687e"
Nov 04 02:15:09.079671 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.079646    4512 scope.go:122] "RemoveContainer" containerID="7f9a742c92b69121ac5eee8529e9021cea83753e89894195f2c72c5a651f719b"
Nov 04 02:15:09.093849 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.093824    4512 scope.go:122] "RemoveContainer" containerID="2f8e6330a1575268869c2e1061e5b55b36ab9cdcb11625587e8d1238dd43f947"
Nov 04 02:15:09.166352 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.166300    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" podStartSLOduration=3.395415194 podStartE2EDuration="9.166285882s" totalImagesPullingTime="5.770870688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:09.164809671 +0000 UTC m=+1170.970266302" watchObservedRunningTime="2026-11-04 02:15:09.166285882 +0000 UTC m=+1170.971742493"
Nov 04 02:15:09.837717 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.837683    4512 scope.go:122] "RemoveContainer" containerID="09c3e7c81dd1d9d3c0f0824738289f9a3b3256fbab0f7d3736ed2406fb716468"
Nov 04 02:15:09.838169 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:09.837998    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8_odh-ai-gateway-infra(764e8d02-1f67-406d-a4b3-fe07edec30e7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7"
Nov 04 02:15:09.839326 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.839302    4512 scope.go:122] "RemoveContainer" containerID="72ef14b4edf480c40b9bd3538383e34115f70c7e3cade09a74744f6dd21509db"
Nov 04 02:15:09.839573 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:09.839553    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895975-fw2zn_odh-ai-gateway-infra(90ae9297-d8ef-454f-9c98-e8adc32e3c93)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93"
Nov 04 02:15:09.840674 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.840657    4512 scope.go:122] "RemoveContainer" containerID="b24e7dc3b6fa0604c2c32d7f3a5bc5dbed418507019dfb1fa5ca744d4fbdb37a"
Nov 04 02:15:09.840883 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:09.840856    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749_odh-ai-gateway-infra(1cd9fb62-802b-4fdc-aafd-4470759d9796)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796"
Nov 04 02:15:09.842202 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.842185    4512 scope.go:122] "RemoveContainer" containerID="b423ca406b7a752816613e107a95d3b162f5acfebdc90f4b432caedf3831e4bb"
Nov 04 02:15:09.842391 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:09.842369    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46_odh-ai-gateway-infra(23c08a30-a814-42a4-bee0-bb5a7e39aa01)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01"
Nov 04 02:15:09.843749 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.843730    4512 generic.go:356] "Generic (PLEG): container finished" podID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerID="3036f4d8384986b69bdfe47d7b66f05d4638dd7c892f521bdece50baf2569c14" exitCode=60
Nov 04 02:15:09.843842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.843758    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" event={"ID":"4b946ba4-63d4-4019-8c5c-7e7f324720e5","Type":"ContainerDied","Data":"3036f4d8384986b69bdfe47d7b66f05d4638dd7c892f521bdece50baf2569c14"}
Nov 04 02:15:09.843842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.843794    4512 scope.go:122] "RemoveContainer" containerID="d1fe5af7f0acc3fd3e84ff19947c27b67807ff582503daf8cdf20d084163bf4f"
Nov 04 02:15:09.843959 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:09.843942    4512 scope.go:122] "RemoveContainer" containerID="3036f4d8384986b69bdfe47d7b66f05d4638dd7c892f521bdece50baf2569c14"
Nov 04 02:15:09.844172 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:09.844151    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv_odh-ai-gateway-infra(4b946ba4-63d4-4019-8c5c-7e7f324720e5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5"
Nov 04 02:15:10.852139 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:10.852106    4512 scope.go:122] "RemoveContainer" containerID="3036f4d8384986b69bdfe47d7b66f05d4638dd7c892f521bdece50baf2569c14"
Nov 04 02:15:10.852564 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:10.852397    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv_odh-ai-gateway-infra(4b946ba4-63d4-4019-8c5c-7e7f324720e5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5"
Nov 04 02:15:11.102961 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.102877    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:11.306347 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.306313    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:11.645183 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.645144    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"]
Nov 04 02:15:11.651741 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.651719    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:15:11.688389 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.688347    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"]
Nov 04 02:15:11.723121 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.723082    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-055a14\""
Nov 04 02:15:11.723576 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.723532    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-055a14-dockercfg-28x9z\""
Nov 04 02:15:11.795230 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.795193    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"]
Nov 04 02:15:11.799834 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.799810    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:15:11.809047 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.808750    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p\" (UID: \"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:15:11.809047 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.808886    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvbcv\" (UniqueName: \"kubernetes.io/projected/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-kube-api-access-kvbcv\") pod \"payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p\" (UID: \"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:15:11.858335 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.858303    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"]
Nov 04 02:15:11.910581 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.910477    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rng59\" (UniqueName: \"kubernetes.io/projected/b549b289-3be5-495c-8caf-df4e1e9934bf-kube-api-access-rng59\") pod \"payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw\" (UID: \"b549b289-3be5-495c-8caf-df4e1e9934bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:15:11.910581 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.910547    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b549b289-3be5-495c-8caf-df4e1e9934bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw\" (UID: \"b549b289-3be5-495c-8caf-df4e1e9934bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:15:11.910812 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.910637    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p\" (UID: \"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:15:11.910812 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.910730    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvbcv\" (UniqueName: \"kubernetes.io/projected/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-kube-api-access-kvbcv\") pod \"payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p\" (UID: \"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:15:11.911586 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.911554    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p\" (UID: \"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:15:11.971795 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:11.971761    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvbcv\" (UniqueName: \"kubernetes.io/projected/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-kube-api-access-kvbcv\") pod \"payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p\" (UID: \"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:15:12.012321 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.012274    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rng59\" (UniqueName: \"kubernetes.io/projected/b549b289-3be5-495c-8caf-df4e1e9934bf-kube-api-access-rng59\") pod \"payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw\" (UID: \"b549b289-3be5-495c-8caf-df4e1e9934bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:15:12.012488 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.012333    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b549b289-3be5-495c-8caf-df4e1e9934bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw\" (UID: \"b549b289-3be5-495c-8caf-df4e1e9934bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:15:12.013396 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.013373    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b549b289-3be5-495c-8caf-df4e1e9934bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw\" (UID: \"b549b289-3be5-495c-8caf-df4e1e9934bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:15:12.074723 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.074692    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rng59\" (UniqueName: \"kubernetes.io/projected/b549b289-3be5-495c-8caf-df4e1e9934bf-kube-api-access-rng59\") pod \"payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw\" (UID: \"b549b289-3be5-495c-8caf-df4e1e9934bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:15:12.117321 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.117285    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:15:12.268118 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.268082    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:15:12.498788 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.498762    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"]
Nov 04 02:15:12.500407 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:15:12.500375    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb549b289_3be5_495c_8caf_df4e1e9934bf.slice/crio-8b52a5e505741d7955a9aa47f745044134b43fd156a272a28b184faa649f32e0 WatchSource:0}: Error finding container 8b52a5e505741d7955a9aa47f745044134b43fd156a272a28b184faa649f32e0: Status 404 returned error can't find the container with id 8b52a5e505741d7955a9aa47f745044134b43fd156a272a28b184faa649f32e0
Nov 04 02:15:12.677760 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:15:12.677723    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b6b1b0_b1b3_48db_aa50_3ee10efcff1b.slice/crio-cc1fa63f0cf16a3a9a1519f95369dbeaa0688152562819579bc2d76ce335be9a WatchSource:0}: Error finding container cc1fa63f0cf16a3a9a1519f95369dbeaa0688152562819579bc2d76ce335be9a: Status 404 returned error can't find the container with id cc1fa63f0cf16a3a9a1519f95369dbeaa0688152562819579bc2d76ce335be9a
Nov 04 02:15:12.677871 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.677846    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"]
Nov 04 02:15:12.867252 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.867216    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw" event={"ID":"b549b289-3be5-495c-8caf-df4e1e9934bf","Type":"ContainerStarted","Data":"a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b"}
Nov 04 02:15:12.867737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.867267    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw" event={"ID":"b549b289-3be5-495c-8caf-df4e1e9934bf","Type":"ContainerStarted","Data":"8b52a5e505741d7955a9aa47f745044134b43fd156a272a28b184faa649f32e0"}
Nov 04 02:15:12.867737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.867397    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:15:12.868899 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.868877    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p" event={"ID":"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b","Type":"ContainerStarted","Data":"1100b7b092d7012ed19ec47bd784a95969f4220b0635bb7266cc93b59b8f8838"}
Nov 04 02:15:12.869007 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.868926    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:15:12.869007 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.868944    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p" event={"ID":"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b","Type":"ContainerStarted","Data":"cc1fa63f0cf16a3a9a1519f95369dbeaa0688152562819579bc2d76ce335be9a"}
Nov 04 02:15:12.986391 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:12.986326    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw" podStartSLOduration=1.986311935 podStartE2EDuration="1.986311935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:12.983170525 +0000 UTC m=+1174.788627157" watchObservedRunningTime="2026-11-04 02:15:12.986311935 +0000 UTC m=+1174.791768602"
Nov 04 02:15:13.101895 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:13.101768    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p" podStartSLOduration=2.101750009 podStartE2EDuration="2.101750009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:13.099133239 +0000 UTC m=+1174.904589871" watchObservedRunningTime="2026-11-04 02:15:13.101750009 +0000 UTC m=+1174.907206640"
Nov 04 02:15:14.076658 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:14.076619    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:15.890114 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:15.890077    4512 generic.go:356] "Generic (PLEG): container finished" podID="6869b102-72dc-4949-b96d-30b1d207d320" containerID="3ba365cbd12354f12f07aedd851446990b587a9497db226460f662a5872cf41e" exitCode=0
Nov 04 02:15:15.890546 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:15.890129    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-mgqgj-m6j9p" event={"ID":"6869b102-72dc-4949-b96d-30b1d207d320","Type":"ContainerDied","Data":"3ba365cbd12354f12f07aedd851446990b587a9497db226460f662a5872cf41e"}
Nov 04 02:15:17.061084 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.061056    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:15:17.168047 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.167959    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-trusted-ca-bundle\") pod \"6869b102-72dc-4949-b96d-30b1d207d320\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") "
Nov 04 02:15:17.168216 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.168070    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-service-ca-bundle\") pod \"6869b102-72dc-4949-b96d-30b1d207d320\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") "
Nov 04 02:15:17.168216 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.168123    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6869b102-72dc-4949-b96d-30b1d207d320-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/6869b102-72dc-4949-b96d-30b1d207d320-archives-path\") pod \"6869b102-72dc-4949-b96d-30b1d207d320\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") "
Nov 04 02:15:17.168334 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.168223    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6869b102-72dc-4949-b96d-30b1d207d320-kube-api-access-dstrt\" (UniqueName: \"kubernetes.io/projected/6869b102-72dc-4949-b96d-30b1d207d320-kube-api-access-dstrt\") pod \"6869b102-72dc-4949-b96d-30b1d207d320\" (UID: \"6869b102-72dc-4949-b96d-30b1d207d320\") "
Nov 04 02:15:17.168473 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.168445    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-trusted-ca-bundle" pod "6869b102-72dc-4949-b96d-30b1d207d320" (UID: "6869b102-72dc-4949-b96d-30b1d207d320"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:15:17.168473 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.168457    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6869b102-72dc-4949-b96d-30b1d207d320-archives-path" pod "6869b102-72dc-4949-b96d-30b1d207d320" (UID: "6869b102-72dc-4949-b96d-30b1d207d320"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:15:17.168578 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.168488    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-service-ca-bundle" pod "6869b102-72dc-4949-b96d-30b1d207d320" (UID: "6869b102-72dc-4949-b96d-30b1d207d320"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:15:17.168678 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.168663    4512 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-trusted-ca-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:17.168712 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.168681    4512 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6869b102-72dc-4949-b96d-30b1d207d320-service-ca-bundle\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:17.168712 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.168690    4512 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/6869b102-72dc-4949-b96d-30b1d207d320-archives-path\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:17.170435 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.170401    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6869b102-72dc-4949-b96d-30b1d207d320-kube-api-access-dstrt" pod "6869b102-72dc-4949-b96d-30b1d207d320" (UID: "6869b102-72dc-4949-b96d-30b1d207d320"). InnerVolumeSpecName "kube-api-access-dstrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:15:17.270163 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.270119    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dstrt\" (UniqueName: \"kubernetes.io/projected/6869b102-72dc-4949-b96d-30b1d207d320-kube-api-access-dstrt\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:17.907305 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.907265    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-mgqgj-m6j9p" event={"ID":"6869b102-72dc-4949-b96d-30b1d207d320","Type":"ContainerDied","Data":"c8a6eb088fc8ff6b64d73bbe709c3b755e4efd456762ef2a1e0cd63b8673b03a"}
Nov 04 02:15:17.907479 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.907323    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8a6eb088fc8ff6b64d73bbe709c3b755e4efd456762ef2a1e0cd63b8673b03a"
Nov 04 02:15:17.907479 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:17.907278    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-mgqgj-m6j9p"
Nov 04 02:15:18.975102 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:18.975072    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:19.179404 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:19.179362    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:19.378621 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:19.378561    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:20.305479 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:20.305449    4512 scope.go:122] "RemoveContainer" containerID="b423ca406b7a752816613e107a95d3b162f5acfebdc90f4b432caedf3831e4bb"
Nov 04 02:15:20.306424 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:20.306221    4512 scope.go:122] "RemoveContainer" containerID="3036f4d8384986b69bdfe47d7b66f05d4638dd7c892f521bdece50baf2569c14"
Nov 04 02:15:20.306871 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:20.306853    4512 scope.go:122] "RemoveContainer" containerID="72ef14b4edf480c40b9bd3538383e34115f70c7e3cade09a74744f6dd21509db"
Nov 04 02:15:20.931418 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:20.931327    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn" event={"ID":"90ae9297-d8ef-454f-9c98-e8adc32e3c93","Type":"ContainerStarted","Data":"4113ad0c559a056a0c40a6a6319228df3bb127adf4b48c83948e39e07db7cc9b"}
Nov 04 02:15:20.933667 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:20.933491    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" event={"ID":"23c08a30-a814-42a4-bee0-bb5a7e39aa01","Type":"ContainerStarted","Data":"e4e955ca3253a31366a74b7b51f0236acb7e589b670ea5f9f35ed7d2ab301b45"}
Nov 04 02:15:20.935698 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:20.935663    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" event={"ID":"4b946ba4-63d4-4019-8c5c-7e7f324720e5","Type":"ContainerStarted","Data":"f555a6751be35760c96c8b4510c4eb76fe0064c991804ba17a00d07b068bf01f"}
Nov 04 02:15:21.050048 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.049985    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn" podStartSLOduration=15.455065197 podStartE2EDuration="21.049970799s" totalImagesPullingTime="5.594905602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:21.048297131 +0000 UTC m=+1182.853753773" watchObservedRunningTime="2026-11-04 02:15:21.049970799 +0000 UTC m=+1182.855427429"
Nov 04 02:15:21.944096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.944022    4512 generic.go:356] "Generic (PLEG): container finished" podID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerID="4113ad0c559a056a0c40a6a6319228df3bb127adf4b48c83948e39e07db7cc9b" exitCode=60
Nov 04 02:15:21.944096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.944072    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn" event={"ID":"90ae9297-d8ef-454f-9c98-e8adc32e3c93","Type":"ContainerDied","Data":"4113ad0c559a056a0c40a6a6319228df3bb127adf4b48c83948e39e07db7cc9b"}
Nov 04 02:15:21.944657 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.944134    4512 scope.go:122] "RemoveContainer" containerID="72ef14b4edf480c40b9bd3538383e34115f70c7e3cade09a74744f6dd21509db"
Nov 04 02:15:21.944657 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.944460    4512 scope.go:122] "RemoveContainer" containerID="4113ad0c559a056a0c40a6a6319228df3bb127adf4b48c83948e39e07db7cc9b"
Nov 04 02:15:21.944774 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:21.944751    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895975-fw2zn_odh-ai-gateway-infra(90ae9297-d8ef-454f-9c98-e8adc32e3c93)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93"
Nov 04 02:15:21.946384 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.946360    4512 generic.go:356] "Generic (PLEG): container finished" podID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerID="e4e955ca3253a31366a74b7b51f0236acb7e589b670ea5f9f35ed7d2ab301b45" exitCode=60
Nov 04 02:15:21.946472 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.946428    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" event={"ID":"23c08a30-a814-42a4-bee0-bb5a7e39aa01","Type":"ContainerDied","Data":"e4e955ca3253a31366a74b7b51f0236acb7e589b670ea5f9f35ed7d2ab301b45"}
Nov 04 02:15:21.946740 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.946725    4512 scope.go:122] "RemoveContainer" containerID="e4e955ca3253a31366a74b7b51f0236acb7e589b670ea5f9f35ed7d2ab301b45"
Nov 04 02:15:21.946966 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:21.946946    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46_odh-ai-gateway-infra(23c08a30-a814-42a4-bee0-bb5a7e39aa01)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01"
Nov 04 02:15:21.948424 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.948403    4512 generic.go:356] "Generic (PLEG): container finished" podID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerID="f555a6751be35760c96c8b4510c4eb76fe0064c991804ba17a00d07b068bf01f" exitCode=60
Nov 04 02:15:21.948532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.948474    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" event={"ID":"4b946ba4-63d4-4019-8c5c-7e7f324720e5","Type":"ContainerDied","Data":"f555a6751be35760c96c8b4510c4eb76fe0064c991804ba17a00d07b068bf01f"}
Nov 04 02:15:21.948730 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.948716    4512 scope.go:122] "RemoveContainer" containerID="f555a6751be35760c96c8b4510c4eb76fe0064c991804ba17a00d07b068bf01f"
Nov 04 02:15:21.948910 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:21.948892    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv_odh-ai-gateway-infra(4b946ba4-63d4-4019-8c5c-7e7f324720e5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5"
Nov 04 02:15:21.957161 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.957140    4512 scope.go:122] "RemoveContainer" containerID="b423ca406b7a752816613e107a95d3b162f5acfebdc90f4b432caedf3831e4bb"
Nov 04 02:15:21.970061 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:21.970045    4512 scope.go:122] "RemoveContainer" containerID="3036f4d8384986b69bdfe47d7b66f05d4638dd7c892f521bdece50baf2569c14"
Nov 04 02:15:22.063446 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:22.063418    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"]
Nov 04 02:15:22.174106 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:22.174079    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"]
Nov 04 02:15:22.290140 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:22.290109    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"]
Nov 04 02:15:23.131565 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.130746    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"
Nov 04 02:15:23.202904 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.202877    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"
Nov 04 02:15:23.207586 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.207533    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"
Nov 04 02:15:23.237004 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.236974    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b946ba4-63d4-4019-8c5c-7e7f324720e5-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b946ba4-63d4-4019-8c5c-7e7f324720e5-maas-api-service-ca\") pod \"4b946ba4-63d4-4019-8c5c-7e7f324720e5\" (UID: \"4b946ba4-63d4-4019-8c5c-7e7f324720e5\") "
Nov 04 02:15:23.237181 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.237168    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b946ba4-63d4-4019-8c5c-7e7f324720e5-kube-api-access-j4sz7\" (UniqueName: \"kubernetes.io/projected/4b946ba4-63d4-4019-8c5c-7e7f324720e5-kube-api-access-j4sz7\") pod \"4b946ba4-63d4-4019-8c5c-7e7f324720e5\" (UID: \"4b946ba4-63d4-4019-8c5c-7e7f324720e5\") "
Nov 04 02:15:23.237364 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.237340    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b946ba4-63d4-4019-8c5c-7e7f324720e5-maas-api-service-ca" pod "4b946ba4-63d4-4019-8c5c-7e7f324720e5" (UID: "4b946ba4-63d4-4019-8c5c-7e7f324720e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:15:23.237589 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.237566    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b946ba4-63d4-4019-8c5c-7e7f324720e5-maas-api-service-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:23.239711 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.239691    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b946ba4-63d4-4019-8c5c-7e7f324720e5-kube-api-access-j4sz7" pod "4b946ba4-63d4-4019-8c5c-7e7f324720e5" (UID: "4b946ba4-63d4-4019-8c5c-7e7f324720e5"). InnerVolumeSpecName "kube-api-access-j4sz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:15:23.305386 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.305350    4512 scope.go:122] "RemoveContainer" containerID="09c3e7c81dd1d9d3c0f0824738289f9a3b3256fbab0f7d3736ed2406fb716468"
Nov 04 02:15:23.305539 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.305442    4512 scope.go:122] "RemoveContainer" containerID="b24e7dc3b6fa0604c2c32d7f3a5bc5dbed418507019dfb1fa5ca744d4fbdb37a"
Nov 04 02:15:23.338150 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.338123    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23c08a30-a814-42a4-bee0-bb5a7e39aa01-kube-api-access-5tm5l\" (UniqueName: \"kubernetes.io/projected/23c08a30-a814-42a4-bee0-bb5a7e39aa01-kube-api-access-5tm5l\") pod \"23c08a30-a814-42a4-bee0-bb5a7e39aa01\" (UID: \"23c08a30-a814-42a4-bee0-bb5a7e39aa01\") "
Nov 04 02:15:23.338275 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.338239    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23c08a30-a814-42a4-bee0-bb5a7e39aa01-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c08a30-a814-42a4-bee0-bb5a7e39aa01-maas-api-service-ca\") pod \"23c08a30-a814-42a4-bee0-bb5a7e39aa01\" (UID: \"23c08a30-a814-42a4-bee0-bb5a7e39aa01\") "
Nov 04 02:15:23.338374 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.338359    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90ae9297-d8ef-454f-9c98-e8adc32e3c93-kube-api-access-76h8j\" (UniqueName: \"kubernetes.io/projected/90ae9297-d8ef-454f-9c98-e8adc32e3c93-kube-api-access-76h8j\") pod \"90ae9297-d8ef-454f-9c98-e8adc32e3c93\" (UID: \"90ae9297-d8ef-454f-9c98-e8adc32e3c93\") "
Nov 04 02:15:23.338450 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.338433    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90ae9297-d8ef-454f-9c98-e8adc32e3c93-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90ae9297-d8ef-454f-9c98-e8adc32e3c93-maas-api-service-ca\") pod \"90ae9297-d8ef-454f-9c98-e8adc32e3c93\" (UID: \"90ae9297-d8ef-454f-9c98-e8adc32e3c93\") "
Nov 04 02:15:23.338650 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.338581    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23c08a30-a814-42a4-bee0-bb5a7e39aa01-maas-api-service-ca" pod "23c08a30-a814-42a4-bee0-bb5a7e39aa01" (UID: "23c08a30-a814-42a4-bee0-bb5a7e39aa01"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:15:23.338812 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.338791    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90ae9297-d8ef-454f-9c98-e8adc32e3c93-maas-api-service-ca" pod "90ae9297-d8ef-454f-9c98-e8adc32e3c93" (UID: "90ae9297-d8ef-454f-9c98-e8adc32e3c93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:15:23.339002 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.338933    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90ae9297-d8ef-454f-9c98-e8adc32e3c93-maas-api-service-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:23.339126 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.339091    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4sz7\" (UniqueName: \"kubernetes.io/projected/4b946ba4-63d4-4019-8c5c-7e7f324720e5-kube-api-access-j4sz7\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:23.339169 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.339130    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c08a30-a814-42a4-bee0-bb5a7e39aa01-maas-api-service-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:23.340649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.340621    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90ae9297-d8ef-454f-9c98-e8adc32e3c93-kube-api-access-76h8j" pod "90ae9297-d8ef-454f-9c98-e8adc32e3c93" (UID: "90ae9297-d8ef-454f-9c98-e8adc32e3c93"). InnerVolumeSpecName "kube-api-access-76h8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:15:23.340755 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.340735    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23c08a30-a814-42a4-bee0-bb5a7e39aa01-kube-api-access-5tm5l" pod "23c08a30-a814-42a4-bee0-bb5a7e39aa01" (UID: "23c08a30-a814-42a4-bee0-bb5a7e39aa01"). InnerVolumeSpecName "kube-api-access-5tm5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:15:23.440131 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.440087    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tm5l\" (UniqueName: \"kubernetes.io/projected/23c08a30-a814-42a4-bee0-bb5a7e39aa01-kube-api-access-5tm5l\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:23.440131 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.440132    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76h8j\" (UniqueName: \"kubernetes.io/projected/90ae9297-d8ef-454f-9c98-e8adc32e3c93-kube-api-access-76h8j\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:23.879150 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.879062    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:15:23.879280 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.879217    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:15:23.884163 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.884138    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:23.968448 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.968422    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"
Nov 04 02:15:23.968628 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.968452    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv" event={"ID":"4b946ba4-63d4-4019-8c5c-7e7f324720e5","Type":"ContainerDied","Data":"e59bfea7fef2de35bc40f2c359c3e58cb9b935675436a407eb1272d0ae103f52"}
Nov 04 02:15:23.968628 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.968524    4512 scope.go:122] "RemoveContainer" containerID="f555a6751be35760c96c8b4510c4eb76fe0064c991804ba17a00d07b068bf01f"
Nov 04 02:15:23.970737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.970711    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8" event={"ID":"764e8d02-1f67-406d-a4b3-fe07edec30e7","Type":"ContainerStarted","Data":"64bec34ff56d8f4e1a6cdb6607af3f161bc2a5e4ca25e4d34a151d9c5a8335db"}
Nov 04 02:15:23.972205 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.972183    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn" event={"ID":"90ae9297-d8ef-454f-9c98-e8adc32e3c93","Type":"ContainerDied","Data":"df4b911230895ddbe23a8eef977c0a92b258138ee434239c14a80d7a2013070e"}
Nov 04 02:15:23.972205 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.972197    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"
Nov 04 02:15:23.974300 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.974274    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" event={"ID":"1cd9fb62-802b-4fdc-aafd-4470759d9796","Type":"ContainerStarted","Data":"c99e963aecf1205976ed80a04436dc9fd8059a6d89a055089a0e57bfdc342f6e"}
Nov 04 02:15:23.975677 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.975661    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"
Nov 04 02:15:23.975677 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.975678    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46" event={"ID":"23c08a30-a814-42a4-bee0-bb5a7e39aa01","Type":"ContainerDied","Data":"d04e06cb2b55e245dade66ecc2616dbe2ab000d8e5a7afb3ca612908aa9587fc"}
Nov 04 02:15:23.979686 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.979669    4512 scope.go:122] "RemoveContainer" containerID="4113ad0c559a056a0c40a6a6319228df3bb127adf4b48c83948e39e07db7cc9b"
Nov 04 02:15:23.989020 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:23.989001    4512 scope.go:122] "RemoveContainer" containerID="e4e955ca3253a31366a74b7b51f0236acb7e589b670ea5f9f35ed7d2ab301b45"
Nov 04 02:15:24.223622 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.223542    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"]
Nov 04 02:15:24.227731 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.227706    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-196e7e-29895975-sn7mv"]
Nov 04 02:15:24.313045 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.313010    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" path="/var/lib/kubelet/pods/4b946ba4-63d4-4019-8c5c-7e7f324720e5/volumes"
Nov 04 02:15:24.335719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.335693    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"]
Nov 04 02:15:24.339879 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.339856    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895975-fw2zn"]
Nov 04 02:15:24.452965 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.452917    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8" podStartSLOduration=18.851423147 podStartE2EDuration="24.452904256s" totalImagesPullingTime="5.601481109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:24.450719589 +0000 UTC m=+1186.256176219" watchObservedRunningTime="2026-11-04 02:15:24.452904256 +0000 UTC m=+1186.258360886"
Nov 04 02:15:24.563072 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.563035    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"]
Nov 04 02:15:24.567317 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.567289    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-a8ef09-29895975-swr46"]
Nov 04 02:15:24.987122 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.987089    4512 generic.go:356] "Generic (PLEG): container finished" podID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerID="64bec34ff56d8f4e1a6cdb6607af3f161bc2a5e4ca25e4d34a151d9c5a8335db" exitCode=60
Nov 04 02:15:24.987292 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.987160    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8" event={"ID":"764e8d02-1f67-406d-a4b3-fe07edec30e7","Type":"ContainerDied","Data":"64bec34ff56d8f4e1a6cdb6607af3f161bc2a5e4ca25e4d34a151d9c5a8335db"}
Nov 04 02:15:24.987292 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.987246    4512 scope.go:122] "RemoveContainer" containerID="09c3e7c81dd1d9d3c0f0824738289f9a3b3256fbab0f7d3736ed2406fb716468"
Nov 04 02:15:24.987610 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.987580    4512 scope.go:122] "RemoveContainer" containerID="64bec34ff56d8f4e1a6cdb6607af3f161bc2a5e4ca25e4d34a151d9c5a8335db"
Nov 04 02:15:24.987902 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:24.987871    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8_odh-ai-gateway-infra(764e8d02-1f67-406d-a4b3-fe07edec30e7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7"
Nov 04 02:15:24.990185 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.990161    4512 generic.go:356] "Generic (PLEG): container finished" podID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerID="c99e963aecf1205976ed80a04436dc9fd8059a6d89a055089a0e57bfdc342f6e" exitCode=60
Nov 04 02:15:24.990289 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.990233    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" event={"ID":"1cd9fb62-802b-4fdc-aafd-4470759d9796","Type":"ContainerDied","Data":"c99e963aecf1205976ed80a04436dc9fd8059a6d89a055089a0e57bfdc342f6e"}
Nov 04 02:15:24.990538 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.990523    4512 scope.go:122] "RemoveContainer" containerID="c99e963aecf1205976ed80a04436dc9fd8059a6d89a055089a0e57bfdc342f6e"
Nov 04 02:15:24.990733 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:24.990716    4512 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749_odh-ai-gateway-infra(1cd9fb62-802b-4fdc-aafd-4470759d9796)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796"
Nov 04 02:15:24.999970 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:24.999954    4512 scope.go:122] "RemoveContainer" containerID="b24e7dc3b6fa0604c2c32d7f3a5bc5dbed418507019dfb1fa5ca744d4fbdb37a"
Nov 04 02:15:25.462471 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:25.462388    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"]
Nov 04 02:15:25.690349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:25.690320    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"]
Nov 04 02:15:26.155046 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.155024    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"
Nov 04 02:15:26.194196 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.194173    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"
Nov 04 02:15:26.272616 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.272550    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1cd9fb62-802b-4fdc-aafd-4470759d9796-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd9fb62-802b-4fdc-aafd-4470759d9796-maas-api-service-ca\") pod \"1cd9fb62-802b-4fdc-aafd-4470759d9796\" (UID: \"1cd9fb62-802b-4fdc-aafd-4470759d9796\") "
Nov 04 02:15:26.272616 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.272605    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cd9fb62-802b-4fdc-aafd-4470759d9796-kube-api-access-tdprf\" (UniqueName: \"kubernetes.io/projected/1cd9fb62-802b-4fdc-aafd-4470759d9796-kube-api-access-tdprf\") pod \"1cd9fb62-802b-4fdc-aafd-4470759d9796\" (UID: \"1cd9fb62-802b-4fdc-aafd-4470759d9796\") "
Nov 04 02:15:26.272883 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.272864    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cd9fb62-802b-4fdc-aafd-4470759d9796-maas-api-service-ca" pod "1cd9fb62-802b-4fdc-aafd-4470759d9796" (UID: "1cd9fb62-802b-4fdc-aafd-4470759d9796"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:15:26.274755 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.274730    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd9fb62-802b-4fdc-aafd-4470759d9796-kube-api-access-tdprf" pod "1cd9fb62-802b-4fdc-aafd-4470759d9796" (UID: "1cd9fb62-802b-4fdc-aafd-4470759d9796"). InnerVolumeSpecName "kube-api-access-tdprf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:15:26.312629 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.312603    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" path="/var/lib/kubelet/pods/23c08a30-a814-42a4-bee0-bb5a7e39aa01/volumes"
Nov 04 02:15:26.313010 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.312997    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" path="/var/lib/kubelet/pods/90ae9297-d8ef-454f-9c98-e8adc32e3c93/volumes"
Nov 04 02:15:26.373584 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.373557    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/764e8d02-1f67-406d-a4b3-fe07edec30e7-kube-api-access-n55c2\" (UniqueName: \"kubernetes.io/projected/764e8d02-1f67-406d-a4b3-fe07edec30e7-kube-api-access-n55c2\") pod \"764e8d02-1f67-406d-a4b3-fe07edec30e7\" (UID: \"764e8d02-1f67-406d-a4b3-fe07edec30e7\") "
Nov 04 02:15:26.373725 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.373614    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/764e8d02-1f67-406d-a4b3-fe07edec30e7-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/764e8d02-1f67-406d-a4b3-fe07edec30e7-maas-api-service-ca\") pod \"764e8d02-1f67-406d-a4b3-fe07edec30e7\" (UID: \"764e8d02-1f67-406d-a4b3-fe07edec30e7\") "
Nov 04 02:15:26.373977 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.373952    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/764e8d02-1f67-406d-a4b3-fe07edec30e7-maas-api-service-ca" pod "764e8d02-1f67-406d-a4b3-fe07edec30e7" (UID: "764e8d02-1f67-406d-a4b3-fe07edec30e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:15:26.374069 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.373997    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd9fb62-802b-4fdc-aafd-4470759d9796-maas-api-service-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:26.374069 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.374010    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdprf\" (UniqueName: \"kubernetes.io/projected/1cd9fb62-802b-4fdc-aafd-4470759d9796-kube-api-access-tdprf\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:26.375702 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.375682    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/764e8d02-1f67-406d-a4b3-fe07edec30e7-kube-api-access-n55c2" pod "764e8d02-1f67-406d-a4b3-fe07edec30e7" (UID: "764e8d02-1f67-406d-a4b3-fe07edec30e7"). InnerVolumeSpecName "kube-api-access-n55c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:15:26.475483 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.475452    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n55c2\" (UniqueName: \"kubernetes.io/projected/764e8d02-1f67-406d-a4b3-fe07edec30e7-kube-api-access-n55c2\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:26.475483 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:26.475483    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/764e8d02-1f67-406d-a4b3-fe07edec30e7-maas-api-service-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:26.779311 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:26.779274    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:15:27.009853 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:27.009824    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"
Nov 04 02:15:27.010011 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:27.009872    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749" event={"ID":"1cd9fb62-802b-4fdc-aafd-4470759d9796","Type":"ContainerDied","Data":"2f8f6a4e9267e5b5c120b25ab753b902a7a1ede2bd3bb474abfde8e91b9a4210"}
Nov 04 02:15:27.010011 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:27.009938    4512 scope.go:122] "RemoveContainer" containerID="c99e963aecf1205976ed80a04436dc9fd8059a6d89a055089a0e57bfdc342f6e"
Nov 04 02:15:27.011612 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:27.011582    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8" event={"ID":"764e8d02-1f67-406d-a4b3-fe07edec30e7","Type":"ContainerDied","Data":"afa0d4cc7c021bf2a4e8fc6ec8991fee53cacbe0c35d031ddd376e21e5cf49d1"}
Nov 04 02:15:27.011741 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:27.011630    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"
Nov 04 02:15:27.019934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:27.019921    4512 scope.go:122] "RemoveContainer" containerID="64bec34ff56d8f4e1a6cdb6607af3f161bc2a5e4ca25e4d34a151d9c5a8335db"
Nov 04 02:15:27.174620 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:27.174597    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"]
Nov 04 02:15:27.181550 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:27.181499    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-24bea9-29895975-wq749"]
Nov 04 02:15:27.299999 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:27.299976    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"]
Nov 04 02:15:27.306328 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:27.306307    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-3bf869-29895975-84fh8"]
Nov 04 02:15:28.314692 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:28.314658    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796" path="/var/lib/kubelet/pods/1cd9fb62-802b-4fdc-aafd-4470759d9796/volumes"
Nov 04 02:15:28.315050 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:28.315040    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7" path="/var/lib/kubelet/pods/764e8d02-1f67-406d-a4b3-fe07edec30e7/volumes"
Nov 04 02:15:28.869830 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:28.869787    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:29.083519 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:29.083482    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:29.303119 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:29.303087    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:30.846232 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:30.846197    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:32.231894 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.231865    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:32.435319 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.435278    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:32.749538 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.748340    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"]
Nov 04 02:15:32.751747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.751718    4512 cpu_manager.go:414] "containerMap: removing container" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerName="cleanup"
Nov 04 02:15:32.751911 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.751898    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerName="cleanup"
Nov 04 02:15:32.752011 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752001    4512 cpu_manager.go:414] "containerMap: removing container" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerName="cleanup"
Nov 04 02:15:32.752072 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752064    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerName="cleanup"
Nov 04 02:15:32.752154 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752146    4512 cpu_manager.go:414] "containerMap: removing container" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerName="cleanup"
Nov 04 02:15:32.752212 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752204    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerName="cleanup"
Nov 04 02:15:32.752313 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752304    4512 cpu_manager.go:414] "containerMap: removing container" podUID="6869b102-72dc-4949-b96d-30b1d207d320" containerName="insights-gathering"
Nov 04 02:15:32.752378 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752369    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6869b102-72dc-4949-b96d-30b1d207d320" containerName="insights-gathering"
Nov 04 02:15:32.752449 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752440    4512 cpu_manager.go:414] "containerMap: removing container" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerName="cleanup"
Nov 04 02:15:32.752535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752524    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerName="cleanup"
Nov 04 02:15:32.752665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752651    4512 cpu_manager.go:414] "containerMap: removing container" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerName="cleanup"
Nov 04 02:15:32.752752 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752740    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerName="cleanup"
Nov 04 02:15:32.752852 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752840    4512 cpu_manager.go:414] "containerMap: removing container" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerName="cleanup"
Nov 04 02:15:32.752928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.752917    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerName="cleanup"
Nov 04 02:15:32.753021 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.753011    4512 cpu_manager.go:414] "containerMap: removing container" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerName="cleanup"
Nov 04 02:15:32.753097 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.753087    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerName="cleanup"
Nov 04 02:15:32.753193 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.753182    4512 cpu_manager.go:414] "containerMap: removing container" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerName="cleanup"
Nov 04 02:15:32.753272 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.753261    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerName="cleanup"
Nov 04 02:15:32.754322 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.754285    4512 cpu_manager.go:414] "containerMap: removing container" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerName="cleanup"
Nov 04 02:15:32.754614 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.754595    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerName="cleanup"
Nov 04 02:15:32.754805 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.754789    4512 cpu_manager.go:414] "containerMap: removing container" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerName="cleanup"
Nov 04 02:15:32.754894 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.754881    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerName="cleanup"
Nov 04 02:15:32.754985 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.754975    4512 cpu_manager.go:414] "containerMap: removing container" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerName="cleanup"
Nov 04 02:15:32.755058 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.755048    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerName="cleanup"
Nov 04 02:15:32.755142 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.755133    4512 cpu_manager.go:414] "containerMap: removing container" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerName="cleanup"
Nov 04 02:15:32.755218 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.755208    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerName="cleanup"
Nov 04 02:15:32.755307 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.755297    4512 cpu_manager.go:414] "containerMap: removing container" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerName="cleanup"
Nov 04 02:15:32.755380 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.755368    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerName="cleanup"
Nov 04 02:15:32.755776 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.755757    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerName="cleanup"
Nov 04 02:15:32.755889 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.755878    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerName="cleanup"
Nov 04 02:15:32.755981 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.755970    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerName="cleanup"
Nov 04 02:15:32.756059 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.756051    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerName="cleanup"
Nov 04 02:15:32.756129 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.756121    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="6869b102-72dc-4949-b96d-30b1d207d320" containerName="insights-gathering"
Nov 04 02:15:32.756216 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.756207    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerName="cleanup"
Nov 04 02:15:32.756288 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.756279    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerName="cleanup"
Nov 04 02:15:32.756367 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.756359    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerName="cleanup"
Nov 04 02:15:32.756434 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.756426    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerName="cleanup"
Nov 04 02:15:32.758544 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.756528    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerName="cleanup"
Nov 04 02:15:32.758544 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.756600    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerName="cleanup"
Nov 04 02:15:32.758544 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.756640    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerName="cleanup"
Nov 04 02:15:32.758544 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.756656    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="90ae9297-d8ef-454f-9c98-e8adc32e3c93" containerName="cleanup"
Nov 04 02:15:32.772792 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.772762    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:15:32.800205 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.800173    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"]
Nov 04 02:15:32.844833 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.844801    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-0508d1\""
Nov 04 02:15:32.845029 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.844930    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-0508d1-dockercfg-nvx8j\""
Nov 04 02:15:32.845414 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.845382    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e551cd-7dc1-4cfc-b295-2ddfec61963a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk\" (UID: \"82e551cd-7dc1-4cfc-b295-2ddfec61963a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:15:32.845577 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.845560    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7sh9\" (UniqueName: \"kubernetes.io/projected/82e551cd-7dc1-4cfc-b295-2ddfec61963a-kube-api-access-l7sh9\") pod \"payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk\" (UID: \"82e551cd-7dc1-4cfc-b295-2ddfec61963a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:15:32.922107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.922065    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"]
Nov 04 02:15:32.923039 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.923016    4512 cpu_manager.go:414] "containerMap: removing container" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerName="cleanup"
Nov 04 02:15:32.923100 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.923044    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerName="cleanup"
Nov 04 02:15:32.923100 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.923091    4512 cpu_manager.go:414] "containerMap: removing container" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerName="cleanup"
Nov 04 02:15:32.923165 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.923099    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cd9fb62-802b-4fdc-aafd-4470759d9796" containerName="cleanup"
Nov 04 02:15:32.923281 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.923266    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b946ba4-63d4-4019-8c5c-7e7f324720e5" containerName="cleanup"
Nov 04 02:15:32.923316 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.923293    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="764e8d02-1f67-406d-a4b3-fe07edec30e7" containerName="cleanup"
Nov 04 02:15:32.923316 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.923303    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="23c08a30-a814-42a4-bee0-bb5a7e39aa01" containerName="cleanup"
Nov 04 02:15:32.927174 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.927154    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:15:32.946621 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.946597    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e551cd-7dc1-4cfc-b295-2ddfec61963a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk\" (UID: \"82e551cd-7dc1-4cfc-b295-2ddfec61963a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:15:32.946735 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.946632    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4v8x\" (UniqueName: \"kubernetes.io/projected/0783f96e-35a1-45a3-906e-8511da03229a-kube-api-access-q4v8x\") pod \"payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp\" (UID: \"0783f96e-35a1-45a3-906e-8511da03229a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:15:32.946776 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.946740    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7sh9\" (UniqueName: \"kubernetes.io/projected/82e551cd-7dc1-4cfc-b295-2ddfec61963a-kube-api-access-l7sh9\") pod \"payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk\" (UID: \"82e551cd-7dc1-4cfc-b295-2ddfec61963a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:15:32.946857 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.946843    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0783f96e-35a1-45a3-906e-8511da03229a-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp\" (UID: \"0783f96e-35a1-45a3-906e-8511da03229a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:15:32.947253 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.947232    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e551cd-7dc1-4cfc-b295-2ddfec61963a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk\" (UID: \"82e551cd-7dc1-4cfc-b295-2ddfec61963a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:15:32.981875 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:32.981849    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"]
Nov 04 02:15:33.008045 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:33.007979    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7sh9\" (UniqueName: \"kubernetes.io/projected/82e551cd-7dc1-4cfc-b295-2ddfec61963a-kube-api-access-l7sh9\") pod \"payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk\" (UID: \"82e551cd-7dc1-4cfc-b295-2ddfec61963a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:15:33.047980 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:33.047941    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4v8x\" (UniqueName: \"kubernetes.io/projected/0783f96e-35a1-45a3-906e-8511da03229a-kube-api-access-q4v8x\") pod \"payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp\" (UID: \"0783f96e-35a1-45a3-906e-8511da03229a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:15:33.048127 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:33.048052    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0783f96e-35a1-45a3-906e-8511da03229a-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp\" (UID: \"0783f96e-35a1-45a3-906e-8511da03229a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:15:33.048688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:33.048666    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0783f96e-35a1-45a3-906e-8511da03229a-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp\" (UID: \"0783f96e-35a1-45a3-906e-8511da03229a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:15:33.087349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:33.087322    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:15:33.110221 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:33.110191    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4v8x\" (UniqueName: \"kubernetes.io/projected/0783f96e-35a1-45a3-906e-8511da03229a-kube-api-access-q4v8x\") pod \"payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp\" (UID: \"0783f96e-35a1-45a3-906e-8511da03229a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:15:33.241634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:33.241603    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:15:33.481989 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:15:33.481945    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e551cd_7dc1_4cfc_b295_2ddfec61963a.slice/crio-16ca56f42952030bd808e4e37f7f9490c18535219a1ca600779893ce6ae758ce WatchSource:0}: Error finding container 16ca56f42952030bd808e4e37f7f9490c18535219a1ca600779893ce6ae758ce: Status 404 returned error can't find the container with id 16ca56f42952030bd808e4e37f7f9490c18535219a1ca600779893ce6ae758ce
Nov 04 02:15:33.501118 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:33.501087    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"]
Nov 04 02:15:33.643174 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:33.643146    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"]
Nov 04 02:15:33.643570 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:15:33.643545    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0783f96e_35a1_45a3_906e_8511da03229a.slice/crio-d7c474dda9402e12535ad0fbe3ade8b7f14dc00a2a0a8deb1af2bee1fe2e96e6 WatchSource:0}: Error finding container d7c474dda9402e12535ad0fbe3ade8b7f14dc00a2a0a8deb1af2bee1fe2e96e6: Status 404 returned error can't find the container with id d7c474dda9402e12535ad0fbe3ade8b7f14dc00a2a0a8deb1af2bee1fe2e96e6
Nov 04 02:15:33.842877 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:33.842574    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:34.078615 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:34.078563    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp" event={"ID":"0783f96e-35a1-45a3-906e-8511da03229a","Type":"ContainerStarted","Data":"774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76"}
Nov 04 02:15:34.078818 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:34.078634    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp" event={"ID":"0783f96e-35a1-45a3-906e-8511da03229a","Type":"ContainerStarted","Data":"d7c474dda9402e12535ad0fbe3ade8b7f14dc00a2a0a8deb1af2bee1fe2e96e6"}
Nov 04 02:15:34.078818 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:34.078667    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:15:34.081202 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:34.081150    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk" event={"ID":"82e551cd-7dc1-4cfc-b295-2ddfec61963a","Type":"ContainerStarted","Data":"3ee1e468c4e1667381baab20c26d5487c9b0e41f81345f51c1e1fe004378c996"}
Nov 04 02:15:34.081202 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:34.081197    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk" event={"ID":"82e551cd-7dc1-4cfc-b295-2ddfec61963a","Type":"ContainerStarted","Data":"16ca56f42952030bd808e4e37f7f9490c18535219a1ca600779893ce6ae758ce"}
Nov 04 02:15:34.081397 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:34.081234    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:15:34.203253 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:34.203156    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp" podStartSLOduration=2.203142007 podStartE2EDuration="2.203142007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:34.197995896 +0000 UTC m=+1196.003452527" watchObservedRunningTime="2026-11-04 02:15:34.203142007 +0000 UTC m=+1196.008598639"
Nov 04 02:15:34.206898 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:34.206870    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:34.316888 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:34.316692    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk" podStartSLOduration=2.316673636 podStartE2EDuration="2.316673636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:34.313219187 +0000 UTC m=+1196.118675848" watchObservedRunningTime="2026-11-04 02:15:34.316673636 +0000 UTC m=+1196.122130271"
Nov 04 02:15:38.639806 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:38.639777    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:15:38.644860 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:38.644838    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:15:39.079595 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:39.079556    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:43.776709 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:43.776673    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:44.000906 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:44.000855    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:44.208107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:44.208004    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:45.090667 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:45.090635    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:15:45.091105 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:45.090717    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:15:48.972481 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:48.972388    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:49.871281 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:49.871239    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"]
Nov 04 02:15:49.888051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:49.888012    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:49.934432 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:49.934376    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"]
Nov 04 02:15:49.978829 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:49.978787    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"distinct-2-w4-196e7e-kserve-self-signed-certs\""
Nov 04 02:15:50.011623 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.011572    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:50.039368 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.039324    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-dshm\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.039830 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.039804    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-tmp-dir\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.040033 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.040015    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp94q\" (UniqueName: \"kubernetes.io/projected/31069c58-7113-4e77-9672-9862fd6576da-kube-api-access-pp94q\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.040254 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.040236    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-model-cache\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.040425 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.040401    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31069c58-7113-4e77-9672-9862fd6576da-tls-certs\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.040767 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.040654    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-home\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.142202 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.142108    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-dshm\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.142399 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.142218    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-tmp-dir\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.142399 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.142249    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp94q\" (UniqueName: \"kubernetes.io/projected/31069c58-7113-4e77-9672-9862fd6576da-kube-api-access-pp94q\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.142399 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.142316    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-model-cache\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.142399 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.142339    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31069c58-7113-4e77-9672-9862fd6576da-tls-certs\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.142399 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.142381    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-home\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.142728 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.142664    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-tmp-dir\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.142728 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.142718    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-model-cache\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.142845 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.142825    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-home\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.144801 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.144771    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-dshm\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.145353 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.145323    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31069c58-7113-4e77-9672-9862fd6576da-tls-certs\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.204026 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.203989    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp94q\" (UniqueName: \"kubernetes.io/projected/31069c58-7113-4e77-9672-9862fd6576da-kube-api-access-pp94q\") pod \"distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.205769 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.205739    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:50.598553 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:50.598473    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"]
Nov 04 02:15:50.599024 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:15:50.598998    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31069c58_7113_4e77_9672_9862fd6576da.slice/crio-a154568ee4234ad0957b8da0ffa8ea316e53e20884903a9bb13e895fc5149c13 WatchSource:0}: Error finding container a154568ee4234ad0957b8da0ffa8ea316e53e20884903a9bb13e895fc5149c13: Status 404 returned error can't find the container with id a154568ee4234ad0957b8da0ffa8ea316e53e20884903a9bb13e895fc5149c13
Nov 04 02:15:51.210952 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:51.210912    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs" event={"ID":"31069c58-7113-4e77-9672-9862fd6576da","Type":"ContainerStarted","Data":"a48a838e35ec1bb5e612db79ddc8ed7c5545bb7e786b4879c3cc45b84fdd88f7"}
Nov 04 02:15:51.211364 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:51.210962    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs" event={"ID":"31069c58-7113-4e77-9672-9862fd6576da","Type":"ContainerStarted","Data":"a154568ee4234ad0957b8da0ffa8ea316e53e20884903a9bb13e895fc5149c13"}
Nov 04 02:15:51.331619 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:51.331560    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs" podStartSLOduration=2.331543998 podStartE2EDuration="2.331543998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:51.330455391 +0000 UTC m=+1213.135912025" watchObservedRunningTime="2026-11-04 02:15:51.331543998 +0000 UTC m=+1213.137000629"
Nov 04 02:15:52.206168 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:52.206127    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:15:53.866939 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:53.866905    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:56.778412 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:15:56.778374    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:15:58.798584 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:15:58.794640    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:00.206188 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:00.206139    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:16:00.220051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:00.220018    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:16:00.292263 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:00.292219    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:16:03.336339 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:03.336293    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:03.537259 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:03.537215    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:03.761048 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:03.761011    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:08.623772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:08.623713    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:08.828104 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:08.828059    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:09.032975 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:09.032933    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:10.811741 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:10.811703    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:11.437642 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:11.437601    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:11.841989 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:11.841948    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:13.710242 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:13.710198    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:15.344909 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:15.344870    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:15.547787 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:15.547745    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:16.919959 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:16.919925    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:17.133216 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:17.133170    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:18.667461 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.667423    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:18.781851 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.781815    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"]
Nov 04 02:16:18.786855 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.786796    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:18.872323 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.872261    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:18.897341 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.897302    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"]
Nov 04 02:16:18.923545 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.923425    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:18.923545 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.923476    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:18.923545 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.923536    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:18.923838 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.923598    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:18.923838 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.923624    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5p8g\" (UniqueName: \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-kube-api-access-n5p8g\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:18.923838 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.923703    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:18.923838 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.923755    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:18.923838 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.923826    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:18.924192 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:18.923879    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.024871 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.024834    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.024871 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.024876    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025146 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.024896    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025146 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.024920    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025146 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.024937    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5p8g\" (UniqueName: \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-kube-api-access-n5p8g\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025146 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.024974    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025146 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.025025    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025146 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.025091    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025146 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.025142    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025550 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.025328    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025550 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.025368    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025677 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.025621    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.025677 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.025652    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.026146 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.026122    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.026695 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.026677    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"]
Nov 04 02:16:19.028035 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.028008    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.028142 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.028039    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.031294 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.031278    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:16:19.079498 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.079460    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:19.085639 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.085606    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-dab533\""
Nov 04 02:16:19.089165 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.089130    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"]
Nov 04 02:16:19.089371 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.089261    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5p8g\" (UniqueName: \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-kube-api-access-n5p8g\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.089647 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.089625    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.093167 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.093142    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-dab533-dockercfg-xpshr\""
Nov 04 02:16:19.105987 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.105953    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:19.129590 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.129551    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m9zc\" (UniqueName: \"kubernetes.io/projected/f5084909-e86c-4257-b2b1-7bef33bf0d15-kube-api-access-8m9zc\") pod \"payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds\" (UID: \"f5084909-e86c-4257-b2b1-7bef33bf0d15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:16:19.129892 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.129866    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5084909-e86c-4257-b2b1-7bef33bf0d15-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds\" (UID: \"f5084909-e86c-4257-b2b1-7bef33bf0d15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:16:19.194225 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.194134    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"]
Nov 04 02:16:19.198960 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.198934    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:16:19.240532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.237227    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5084909-e86c-4257-b2b1-7bef33bf0d15-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds\" (UID: \"f5084909-e86c-4257-b2b1-7bef33bf0d15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:16:19.240532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.237466    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m9zc\" (UniqueName: \"kubernetes.io/projected/f5084909-e86c-4257-b2b1-7bef33bf0d15-kube-api-access-8m9zc\") pod \"payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds\" (UID: \"f5084909-e86c-4257-b2b1-7bef33bf0d15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:16:19.240532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.238745    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5084909-e86c-4257-b2b1-7bef33bf0d15-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds\" (UID: \"f5084909-e86c-4257-b2b1-7bef33bf0d15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:16:19.240532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.239711    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"]
Nov 04 02:16:19.299152 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.299116    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m9zc\" (UniqueName: \"kubernetes.io/projected/f5084909-e86c-4257-b2b1-7bef33bf0d15-kube-api-access-8m9zc\") pod \"payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds\" (UID: \"f5084909-e86c-4257-b2b1-7bef33bf0d15\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:16:19.338862 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.338818    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9qbd\" (UniqueName: \"kubernetes.io/projected/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-kube-api-access-q9qbd\") pod \"payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh\" (UID: \"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:16:19.339033 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.338897    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh\" (UID: \"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:16:19.364422 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.364378    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:16:19.373593 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.373560    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"]
Nov 04 02:16:19.379022 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.379000    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440247    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440319    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440364    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440419    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93902766-a145-4aa9-bd44-e7eb794347b8-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440469    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzb6s\" (UniqueName: \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-kube-api-access-jzb6s\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440552    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9qbd\" (UniqueName: \"kubernetes.io/projected/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-kube-api-access-q9qbd\") pod \"payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh\" (UID: \"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440604    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh\" (UID: \"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440648    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440674    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440716    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.440748    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93902766-a145-4aa9-bd44-e7eb794347b8-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.442662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.441970    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh\" (UID: \"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:16:19.443495 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.443470    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"]
Nov 04 02:16:19.501662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.501618    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9qbd\" (UniqueName: \"kubernetes.io/projected/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-kube-api-access-q9qbd\") pod \"payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh\" (UID: \"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:16:19.518560 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.518532    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-196e7e-openshift-default-dockercfg-cdj7c\""
Nov 04 02:16:19.520742 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.520718    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:16:19.541533 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.541475    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.541633 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.541548    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.541633 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.541578    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.541633 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.541596    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93902766-a145-4aa9-bd44-e7eb794347b8-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.541805 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.541636    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.541805 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.541703    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.541805 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.541757    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.541969 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.541822    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93902766-a145-4aa9-bd44-e7eb794347b8-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.541969 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.541873    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzb6s\" (UniqueName: \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-kube-api-access-jzb6s\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.541969 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.541931    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.542133 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.542033    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.542229 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.542200    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.542434 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.542398    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.542608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.542588    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93902766-a145-4aa9-bd44-e7eb794347b8-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.543974 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.543952    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.544291 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.544273    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93902766-a145-4aa9-bd44-e7eb794347b8-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.596736 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.591964    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"]
Nov 04 02:16:19.597879 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:16:19.597842    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b19828f_975e_4ed7_8963_29bbb5eb94ca.slice/crio-79a689160924346dbfdf33566ae903fb670d914d5ce3f4ffb4f74b3fdb7ac057 WatchSource:0}: Error finding container 79a689160924346dbfdf33566ae903fb670d914d5ce3f4ffb4f74b3fdb7ac057: Status 404 returned error can't find the container with id 79a689160924346dbfdf33566ae903fb670d914d5ce3f4ffb4f74b3fdb7ac057
Nov 04 02:16:19.602541 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.602457    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:16:19.602633 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.602562    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:16:19.602633 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.602609    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:16:19.603876 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.603824    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzb6s\" (UniqueName: \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-kube-api-access-jzb6s\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.609983 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.609880    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.652401 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.652359    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:19.694340 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.694316    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:19.764267 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.764212    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"]
Nov 04 02:16:19.862564 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.861673    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:19.909885 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:19.909853    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"]
Nov 04 02:16:20.103944 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.103906    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"]
Nov 04 02:16:20.105305 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:16:20.105275    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93902766_a145_4aa9_bd44_e7eb794347b8.slice/crio-6a99eef57e14b8f2bc2ff126247774ed8b4c34bb9d5ac7c9b3b7e29d42e73221 WatchSource:0}: Error finding container 6a99eef57e14b8f2bc2ff126247774ed8b4c34bb9d5ac7c9b3b7e29d42e73221: Status 404 returned error can't find the container with id 6a99eef57e14b8f2bc2ff126247774ed8b4c34bb9d5ac7c9b3b7e29d42e73221
Nov 04 02:16:20.108437 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.108389    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:16:20.108640 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.108476    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:16:20.108640 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.108605    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:16:20.467595 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.466720    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh" event={"ID":"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf","Type":"ContainerStarted","Data":"32f0727ca695d3a8cca867c090bf7e1a2a55112b69dcd28d3c3bc9ca48f2df92"}
Nov 04 02:16:20.467595 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.466801    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:16:20.467595 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.466826    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh" event={"ID":"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf","Type":"ContainerStarted","Data":"3f055e052fde88a98fc947b75bbadef2e67f6ae9f2492ce2cc4992d5b1638b30"}
Nov 04 02:16:20.471529 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.468908    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" event={"ID":"4b19828f-975e-4ed7-8963-29bbb5eb94ca","Type":"ContainerStarted","Data":"049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3"}
Nov 04 02:16:20.471529 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.468966    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" event={"ID":"4b19828f-975e-4ed7-8963-29bbb5eb94ca","Type":"ContainerStarted","Data":"79a689160924346dbfdf33566ae903fb670d914d5ce3f4ffb4f74b3fdb7ac057"}
Nov 04 02:16:20.483532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.481925    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" event={"ID":"93902766-a145-4aa9-bd44-e7eb794347b8","Type":"ContainerStarted","Data":"65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9"}
Nov 04 02:16:20.483532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.481981    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" event={"ID":"93902766-a145-4aa9-bd44-e7eb794347b8","Type":"ContainerStarted","Data":"6a99eef57e14b8f2bc2ff126247774ed8b4c34bb9d5ac7c9b3b7e29d42e73221"}
Nov 04 02:16:20.487537 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.485075    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds" event={"ID":"f5084909-e86c-4257-b2b1-7bef33bf0d15","Type":"ContainerStarted","Data":"d22d38d687d6be50f36ae5ab7352c6bc2f936a1111724d4913ea4a3ebbc848e8"}
Nov 04 02:16:20.487537 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.485138    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds" event={"ID":"f5084909-e86c-4257-b2b1-7bef33bf0d15","Type":"ContainerStarted","Data":"9884a1ab9eec9173e7ae834ed9040e63c30f01b58636e873a4fa2ba4d778e5f4"}
Nov 04 02:16:20.487537 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.486187    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:16:20.625921 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.615041    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh" podStartSLOduration=1.6149766620000001 podStartE2EDuration="1.614976662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:20.600956148 +0000 UTC m=+1242.406412780" watchObservedRunningTime="2026-11-04 02:16:20.614976662 +0000 UTC m=+1242.420433295"
Nov 04 02:16:20.696663 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.696623    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:20.861901 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.861837    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" podStartSLOduration=2.861819435 podStartE2EDuration="2.861819435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:20.857954179 +0000 UTC m=+1242.663410812" watchObservedRunningTime="2026-11-04 02:16:20.861819435 +0000 UTC m=+1242.667276066"
Nov 04 02:16:20.862100 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.861987    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" podStartSLOduration=1.861981163 podStartE2EDuration="1.861981163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:20.734857079 +0000 UTC m=+1242.540313712" watchObservedRunningTime="2026-11-04 02:16:20.861981163 +0000 UTC m=+1242.667437794"
Nov 04 02:16:20.977980 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:20.977906    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds" podStartSLOduration=2.9778847219999998 podStartE2EDuration="2.977884722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:20.973850575 +0000 UTC m=+1242.779307206" watchObservedRunningTime="2026-11-04 02:16:20.977884722 +0000 UTC m=+1242.783341357"
Nov 04 02:16:21.110549 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:21.106856    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:21.117864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:21.117765    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 02:16:21.118134 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:21.118097    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 02:16:21.254856 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:21.254815    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:21.697411 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:21.697366    4512 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 04 02:16:21.698039 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:21.698006    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 04 02:16:22.110089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.110045    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 02:16:22.110405 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.110373    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 02:16:22.371028 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.370922    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-65c44bf774-5666d"]
Nov 04 02:16:22.376856 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.376818    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.444536 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.443959    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-65c44bf774-5666d"]
Nov 04 02:16:22.473951 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.473600    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-plugins-config-volume\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.473951 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.473780    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-routing-overlay-volume\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.473951 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.473857    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgk2l\" (UniqueName: \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-kube-api-access-mgk2l\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.473951 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.473914    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-provider-credentials\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.487068 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.487029    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Nov 04 02:16:22.487068 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.487053    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-26d257c38a93\""
Nov 04 02:16:22.487315 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.487114    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Nov 04 02:16:22.487315 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.487160    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Nov 04 02:16:22.487315 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.487246    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-5wqjh\""
Nov 04 02:16:22.487583 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.487560    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Nov 04 02:16:22.575524 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.575460    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgk2l\" (UniqueName: \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-kube-api-access-mgk2l\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.575715 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.575567    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-provider-credentials\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.575715 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.575675    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-plugins-config-volume\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.575788 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.575728    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-routing-overlay-volume\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.576337 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.576313    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-routing-overlay-volume\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.576473 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.576448    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-plugins-config-volume\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.578367 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.578338    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-provider-credentials\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.640593 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.639825    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgk2l\" (UniqueName: \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-kube-api-access-mgk2l\") pod \"payload-processing-external-model-65c44bf774-5666d\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") " pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:22.703547 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.703052    4512 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 04 02:16:22.703547 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.703134    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 04 02:16:22.720543 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:22.719767    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:23.152539 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:23.151816    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 02:16:23.152539 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:23.151896    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 02:16:23.250176 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:23.250143    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-65c44bf774-5666d"]
Nov 04 02:16:23.572736 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:23.572704    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d" event={"ID":"21ce36d2-f8ab-4895-b3f5-9630ca4280f3","Type":"ContainerStarted","Data":"3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93"}
Nov 04 02:16:23.572898 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:23.572757    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d" event={"ID":"21ce36d2-f8ab-4895-b3f5-9630ca4280f3","Type":"ContainerStarted","Data":"7da51bbc04f4c8aeb2eb82d581506e4b7f7d11bbf9a71c6ff6a74eb16085f49a"}
Nov 04 02:16:23.574561 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:23.573492    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:23.707609 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:23.700260    4512 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 04 02:16:23.707609 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:23.700325    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 04 02:16:24.108101 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:24.108055    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 02:16:24.108432 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:24.108390    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 02:16:24.711120 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:24.711074    4512 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 04 02:16:24.712716 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:24.712677    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 04 02:16:24.876086 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:24.876058    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:25.116535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:25.116016    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 02:16:25.116535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:25.116093    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 02:16:25.699535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:25.695970    4512 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 04 02:16:25.699535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:25.696039    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 04 02:16:26.106965 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.106923    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 02:16:26.107814 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.107778    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 02:16:26.141499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.141427    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d" podStartSLOduration=4.141408354 podStartE2EDuration="4.141408354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:23.699832393 +0000 UTC m=+1245.505289026" watchObservedRunningTime="2026-11-04 02:16:26.141408354 +0000 UTC m=+1247.946864987"
Nov 04 02:16:26.176086 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.176046    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"]
Nov 04 02:16:26.181865 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.181833    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.236532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.220936    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"]
Nov 04 02:16:26.249224 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.249187    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-ee7ade4978e6\""
Nov 04 02:16:26.277193 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.277140    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-routing-overlay-volume\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.277381 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.277201    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.277381 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.277269    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfbh2\" (UniqueName: \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-kube-api-access-mfbh2\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.277381 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.277313    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-plugins-config-volume\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.378718 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.378608    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-routing-overlay-volume\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.378718 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.378685    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.378990 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.378775    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfbh2\" (UniqueName: \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-kube-api-access-mfbh2\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.378990 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.378825    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-plugins-config-volume\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.379741 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.379715    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-plugins-config-volume\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.379873 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.379812    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-routing-overlay-volume\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.382388 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.382356    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.440992 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.440964    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfbh2\" (UniqueName: \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-kube-api-access-mfbh2\") pod \"payload-processing-external-model-6945d56d5f-5trg4\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") " pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.534020 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.533977    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:26.710533 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.702059    4512 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 04 02:16:26.710533 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.702122    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 04 02:16:26.819459 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:16:26.819321    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:16:26.966811 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:26.966767    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"]
Nov 04 02:16:26.985028 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:16:26.983935    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3300f6_b801_481d_a7f9_b76114860e85.slice/crio-b0efc945f58c280a6773a2457aed0f4ec313768b2affe131a993404c16ccc73f WatchSource:0}: Error finding container b0efc945f58c280a6773a2457aed0f4ec313768b2affe131a993404c16ccc73f: Status 404 returned error can't find the container with id b0efc945f58c280a6773a2457aed0f4ec313768b2affe131a993404c16ccc73f
Nov 04 02:16:27.114662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:27.114138    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 02:16:27.114662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:27.114200    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 02:16:27.635860 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:27.635823    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4" event={"ID":"1d3300f6-b801-481d-a7f9-b76114860e85","Type":"ContainerStarted","Data":"7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe"}
Nov 04 02:16:27.636070 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:27.635886    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4" event={"ID":"1d3300f6-b801-481d-a7f9-b76114860e85","Type":"ContainerStarted","Data":"b0efc945f58c280a6773a2457aed0f4ec313768b2affe131a993404c16ccc73f"}
Nov 04 02:16:27.636070 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:27.636012    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:27.695365 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:27.695327    4512 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 04 02:16:27.695559 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:27.695399    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 04 02:16:27.756438 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:27.756367    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4" podStartSLOduration=1.75634889 podStartE2EDuration="1.75634889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:27.754643363 +0000 UTC m=+1249.560099996" watchObservedRunningTime="2026-11-04 02:16:27.75634889 +0000 UTC m=+1249.561805523"
Nov 04 02:16:28.158086 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:28.158055    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:28.158594 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:28.158569    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:28.159641 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:28.159618    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:16:28.958177 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:28.958142    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:28.958676 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:28.958641    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:28.959668 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:28.959647    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:16:30.197086 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:30.197045    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:31.228273 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:31.228237    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:31.450784 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:31.450742    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:31.526653 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:31.526619    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:16:31.530746 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:31.530717    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:16:35.082370 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:35.082333    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:35.590946 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:35.590910    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:38.650756 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:38.650728    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:16:38.823753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:38.823720    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-65c44bf774-5666d"]
Nov 04 02:16:39.571973 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:39.571926    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:40.233806 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:40.233763    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:40.943679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:40.943646    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:41.173698 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:41.173666    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:41.376595 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:41.376555    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:41.588917 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:41.588875    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:41.814114 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:41.814079    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:42.565209 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:42.565172    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:42.770049 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:42.770013    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:43.825168 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:43.825126    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d" podUID="21ce36d2-f8ab-4895-b3f5-9630ca4280f3" containerName="payload-processing" containerID="cri-o://3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93" gracePeriod=25
Nov 04 02:16:44.108434 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.108404    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:44.207960 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.207918    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-kube-api-access-mgk2l\" (UniqueName: \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-kube-api-access-mgk2l\") pod \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") "
Nov 04 02:16:44.208118 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.207968    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-plugins-config-volume\") pod \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") "
Nov 04 02:16:44.208118 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.208044    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-routing-overlay-volume\") pod \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") "
Nov 04 02:16:44.208210 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.208164    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-provider-credentials\" (UniqueName: \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-provider-credentials\") pod \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\" (UID: \"21ce36d2-f8ab-4895-b3f5-9630ca4280f3\") "
Nov 04 02:16:44.208450 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.208421    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-plugins-config-volume" pod "21ce36d2-f8ab-4895-b3f5-9630ca4280f3" (UID: "21ce36d2-f8ab-4895-b3f5-9630ca4280f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:16:44.208551 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.208457    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-routing-overlay-volume" pod "21ce36d2-f8ab-4895-b3f5-9630ca4280f3" (UID: "21ce36d2-f8ab-4895-b3f5-9630ca4280f3"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:16:44.208761 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.208712    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:16:44.208761 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.208730    4512 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-routing-overlay-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:16:44.210487 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.210461    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-kube-api-access-mgk2l" pod "21ce36d2-f8ab-4895-b3f5-9630ca4280f3" (UID: "21ce36d2-f8ab-4895-b3f5-9630ca4280f3"). InnerVolumeSpecName "kube-api-access-mgk2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:16:44.210487 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.210475    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-provider-credentials" pod "21ce36d2-f8ab-4895-b3f5-9630ca4280f3" (UID: "21ce36d2-f8ab-4895-b3f5-9630ca4280f3"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:16:44.309440 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.309406    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgk2l\" (UniqueName: \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-kube-api-access-mgk2l\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:16:44.309440 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.309437    4512 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/21ce36d2-f8ab-4895-b3f5-9630ca4280f3-provider-credentials\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:16:44.774873 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.774837    4512 generic.go:356] "Generic (PLEG): container finished" podID="21ce36d2-f8ab-4895-b3f5-9630ca4280f3" containerID="3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93" exitCode=0
Nov 04 02:16:44.775082 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.774894    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d"
Nov 04 02:16:44.775082 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.774926    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d" event={"ID":"21ce36d2-f8ab-4895-b3f5-9630ca4280f3","Type":"ContainerDied","Data":"3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93"}
Nov 04 02:16:44.775082 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.774983    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-65c44bf774-5666d" event={"ID":"21ce36d2-f8ab-4895-b3f5-9630ca4280f3","Type":"ContainerDied","Data":"7da51bbc04f4c8aeb2eb82d581506e4b7f7d11bbf9a71c6ff6a74eb16085f49a"}
Nov 04 02:16:44.775082 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.775009    4512 scope.go:122] "RemoveContainer" containerID="3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93"
Nov 04 02:16:44.785282 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.785261    4512 scope.go:122] "RemoveContainer" containerID="3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93"
Nov 04 02:16:44.785577 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:16:44.785552    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93\": container with ID starting with 3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93 not found: ID does not exist" containerID="3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93"
Nov 04 02:16:44.785633 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.785583    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93"} err="failed to get container status \"3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93\": rpc error: code = NotFound desc = could not find container \"3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93\": container with ID starting with 3bd67addffd4c59896d40cb673c49355920af3555270985d21d6e5bd35fe9b93 not found: ID does not exist"
Nov 04 02:16:44.942027 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.941986    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-65c44bf774-5666d"]
Nov 04 02:16:44.996189 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:44.996146    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-65c44bf774-5666d"]
Nov 04 02:16:45.470245 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:45.470210    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:46.314710 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:46.314674    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21ce36d2-f8ab-4895-b3f5-9630ca4280f3" path="/var/lib/kubelet/pods/21ce36d2-f8ab-4895-b3f5-9630ca4280f3/volumes"
Nov 04 02:16:46.990748 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:46.990711    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:48.096367 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.096327    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"]
Nov 04 02:16:48.097349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.097326    4512 cpu_manager.go:414] "containerMap: removing container" podUID="21ce36d2-f8ab-4895-b3f5-9630ca4280f3" containerName="payload-processing"
Nov 04 02:16:48.097398 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.097353    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21ce36d2-f8ab-4895-b3f5-9630ca4280f3" containerName="payload-processing"
Nov 04 02:16:48.097610 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.097596    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="21ce36d2-f8ab-4895-b3f5-9630ca4280f3" containerName="payload-processing"
Nov 04 02:16:48.103343 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.103316    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.156829 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.156795    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.157004 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.156909    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.157004 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.156963    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.157004 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.156996    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhlgm\" (UniqueName: \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-kube-api-access-xhlgm\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.157195 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.157180    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-055a14\"/\"kube-root-ca.crt\""
Nov 04 02:16:48.159185 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.159150    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"]
Nov 04 02:16:48.203143 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.203103    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-055a14\"/\"agc-cred-dc9c3b71fc87\""
Nov 04 02:16:48.203143 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.203140    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-055a14\"/\"routing-overlay\""
Nov 04 02:16:48.203143 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.203103    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-055a14\"/\"payload-processing-external-model-plugins-e2e-extmod-b-055a14\""
Nov 04 02:16:48.203143 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.203108    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-055a14\"/\"openshift-service-ca.crt\""
Nov 04 02:16:48.203433 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.203112    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-055a14\"/\"payload-processing-external-model-e2e--498d691e-dockercfg-nm6b4\""
Nov 04 02:16:48.258629 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.258581    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.258793 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.258656    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.258793 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.258689    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhlgm\" (UniqueName: \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-kube-api-access-xhlgm\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.258793 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.258742    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.259499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.259465    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.259638 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.259570    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.261399 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.261372    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.320319 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.320284    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhlgm\" (UniqueName: \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-kube-api-access-xhlgm\") pod \"payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") " pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.419802 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.419707    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:48.808763 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:48.808735    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"]
Nov 04 02:16:48.811658 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:16:48.811616    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0569531_49c1_412f_91d0_053f66157457.slice/crio-0f741fb105b7408f8947bbd37e23a7f02612554c959fd4d8e4ab3152fee1d8a9 WatchSource:0}: Error finding container 0f741fb105b7408f8947bbd37e23a7f02612554c959fd4d8e4ab3152fee1d8a9: Status 404 returned error can't find the container with id 0f741fb105b7408f8947bbd37e23a7f02612554c959fd4d8e4ab3152fee1d8a9
Nov 04 02:16:49.821813 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:49.821760    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw" event={"ID":"c0569531-49c1-412f-91d0-053f66157457","Type":"ContainerStarted","Data":"8f6dce22dd517afdcb82664d007b6cfc963d5f228526d2b49c6de13dc5f48ba6"}
Nov 04 02:16:49.822338 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:49.821848    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw" event={"ID":"c0569531-49c1-412f-91d0-053f66157457","Type":"ContainerStarted","Data":"0f741fb105b7408f8947bbd37e23a7f02612554c959fd4d8e4ab3152fee1d8a9"}
Nov 04 02:16:49.822338 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:49.821878    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:16:49.941655 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:49.941596    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw" podStartSLOduration=1.9415806249999998 podStartE2EDuration="1.941580625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:49.938444924 +0000 UTC m=+1271.743901558" watchObservedRunningTime="2026-11-04 02:16:49.941580625 +0000 UTC m=+1271.747037257"
Nov 04 02:16:50.381167 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:50.381120    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:55.412884 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:16:55.412848    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:56.786239 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:16:56.786202    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:17:00.228215 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:00.228176    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:00.483196 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:00.483157    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:00.686251 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:00.686212    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:00.836306 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:00.836213    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:17:05.558040 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:05.557999    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:05.810251 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:05.810146    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:07.047194 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.047154    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:07.371757 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.371670    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"]
Nov 04 02:17:07.375706 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.375686    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.437092 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.437048    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"]
Nov 04 02:17:07.478892 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.478863    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:17:07.478892 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.478863    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:17:07.502171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.502135    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9c237ef8-tgn2t\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.502354 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.502204    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9c237ef8-tgn2t\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.502445 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.502424    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm6wg\" (UniqueName: \"kubernetes.io/projected/26be670d-5b5e-4a6b-b623-ac721ceb4fda-kube-api-access-mm6wg\") pod \"maas-api-revoke-sub-9c237ef8-tgn2t\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.604569 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.604262    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9c237ef8-tgn2t\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.604569 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.604353    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9c237ef8-tgn2t\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.604569 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.604486    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm6wg\" (UniqueName: \"kubernetes.io/projected/26be670d-5b5e-4a6b-b623-ac721ceb4fda-kube-api-access-mm6wg\") pod \"maas-api-revoke-sub-9c237ef8-tgn2t\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.609875 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.609845    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9c237ef8-tgn2t\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.655604 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.653568    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9c237ef8-tgn2t\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.670480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.670040    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm6wg\" (UniqueName: \"kubernetes.io/projected/26be670d-5b5e-4a6b-b623-ac721ceb4fda-kube-api-access-mm6wg\") pod \"maas-api-revoke-sub-9c237ef8-tgn2t\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.691737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.691700    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:07.703819 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:07.703698    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:08.030932 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:08.030851    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:08.124490 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:08.124458    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"]
Nov 04 02:17:08.125735 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:17:08.125702    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26be670d_5b5e_4a6b_b623_ac721ceb4fda.slice/crio-dab5711b3bf9b89a809f359460394cf59f91474642d58285782df3cd97c12390 WatchSource:0}: Error finding container dab5711b3bf9b89a809f359460394cf59f91474642d58285782df3cd97c12390: Status 404 returned error can't find the container with id dab5711b3bf9b89a809f359460394cf59f91474642d58285782df3cd97c12390
Nov 04 02:17:08.996708 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:08.996669    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t" event={"ID":"26be670d-5b5e-4a6b-b623-ac721ceb4fda","Type":"ContainerStarted","Data":"3c8c5b1d2fe291ee12bc1b29a90f2033c548e5fa466c3d000bc878efdaa18884"}
Nov 04 02:17:08.996921 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:08.996734    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t" event={"ID":"26be670d-5b5e-4a6b-b623-ac721ceb4fda","Type":"ContainerStarted","Data":"dab5711b3bf9b89a809f359460394cf59f91474642d58285782df3cd97c12390"}
Nov 04 02:17:09.114760 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:09.114701    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t" podStartSLOduration=2.114681244 podStartE2EDuration="2.114681244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:09.112112403 +0000 UTC m=+1290.917569062" watchObservedRunningTime="2026-11-04 02:17:09.114681244 +0000 UTC m=+1290.920137877"
Nov 04 02:17:10.006256 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:10.006218    4512 generic.go:356] "Generic (PLEG): container finished" podID="26be670d-5b5e-4a6b-b623-ac721ceb4fda" containerID="3c8c5b1d2fe291ee12bc1b29a90f2033c548e5fa466c3d000bc878efdaa18884" exitCode=0
Nov 04 02:17:10.006701 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:10.006309    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t" event={"ID":"26be670d-5b5e-4a6b-b623-ac721ceb4fda","Type":"ContainerDied","Data":"3c8c5b1d2fe291ee12bc1b29a90f2033c548e5fa466c3d000bc878efdaa18884"}
Nov 04 02:17:10.720676 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:10.720641    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:11.162388 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.162363    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:11.261549 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.261340    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26be670d-5b5e-4a6b-b623-ac721ceb4fda-kube-api-access-mm6wg\" (UniqueName: \"kubernetes.io/projected/26be670d-5b5e-4a6b-b623-ac721ceb4fda-kube-api-access-mm6wg\") pod \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") "
Nov 04 02:17:11.261549 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.261604    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-openshift-ca\") pod \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") "
Nov 04 02:17:11.261549 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.261803    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-xks-ca\") pod \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\" (UID: \"26be670d-5b5e-4a6b-b623-ac721ceb4fda\") "
Nov 04 02:17:11.262173 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.262140    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-openshift-ca" pod "26be670d-5b5e-4a6b-b623-ac721ceb4fda" (UID: "26be670d-5b5e-4a6b-b623-ac721ceb4fda"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:17:11.262477 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.262443    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:11.265075 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.265018    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26be670d-5b5e-4a6b-b623-ac721ceb4fda-kube-api-access-mm6wg" pod "26be670d-5b5e-4a6b-b623-ac721ceb4fda" (UID: "26be670d-5b5e-4a6b-b623-ac721ceb4fda"). InnerVolumeSpecName "kube-api-access-mm6wg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:17:11.265599 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.265581    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-xks-ca" pod "26be670d-5b5e-4a6b-b623-ac721ceb4fda" (UID: "26be670d-5b5e-4a6b-b623-ac721ceb4fda"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:17:11.364522 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.364469    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm6wg\" (UniqueName: \"kubernetes.io/projected/26be670d-5b5e-4a6b-b623-ac721ceb4fda-kube-api-access-mm6wg\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:11.364522 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.364520    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26be670d-5b5e-4a6b-b623-ac721ceb4fda-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:11.992557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:11.992485    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:12.024809 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:12.024780    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"
Nov 04 02:17:12.024809 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:12.024804    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t" event={"ID":"26be670d-5b5e-4a6b-b623-ac721ceb4fda","Type":"ContainerDied","Data":"dab5711b3bf9b89a809f359460394cf59f91474642d58285782df3cd97c12390"}
Nov 04 02:17:12.024993 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:12.024869    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dab5711b3bf9b89a809f359460394cf59f91474642d58285782df3cd97c12390"
Nov 04 02:17:12.214714 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:12.214681    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:16.064362 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.064321    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:16.312937 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.312904    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:16.413182 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.413077    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"]
Nov 04 02:17:16.414847 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.414822    4512 cpu_manager.go:414] "containerMap: removing container" podUID="26be670d-5b5e-4a6b-b623-ac721ceb4fda" containerName="revoke-keys"
Nov 04 02:17:16.414847 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.414846    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26be670d-5b5e-4a6b-b623-ac721ceb4fda" containerName="revoke-keys"
Nov 04 02:17:16.415192 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.415174    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="26be670d-5b5e-4a6b-b623-ac721ceb4fda" containerName="revoke-keys"
Nov 04 02:17:16.425005 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.424972    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:17:16.476120 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.476081    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"]
Nov 04 02:17:16.521789 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.521756    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-3f76d0-dockercfg-9ktlm\""
Nov 04 02:17:16.522156 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.522139    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-3f76d0\""
Nov 04 02:17:16.553480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.553439    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx7nk\" (UniqueName: \"kubernetes.io/projected/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-kube-api-access-vx7nk\") pod \"payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv\" (UID: \"1d0189f7-fe9e-44a5-bcd5-24f125a267ff\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:17:16.553956 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.553913    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv\" (UID: \"1d0189f7-fe9e-44a5-bcd5-24f125a267ff\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:17:16.573882 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.573850    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"]
Nov 04 02:17:16.578413 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.578390    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:17:16.623909 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.623871    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"]
Nov 04 02:17:16.655474 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.655423    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx7nk\" (UniqueName: \"kubernetes.io/projected/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-kube-api-access-vx7nk\") pod \"payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv\" (UID: \"1d0189f7-fe9e-44a5-bcd5-24f125a267ff\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:17:16.655699 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.655479    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a8c919-a691-4100-8472-cab6a495f733-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq\" (UID: \"a1a8c919-a691-4100-8472-cab6a495f733\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:17:16.655770 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.655720    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfvjx\" (UniqueName: \"kubernetes.io/projected/a1a8c919-a691-4100-8472-cab6a495f733-kube-api-access-pfvjx\") pod \"payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq\" (UID: \"a1a8c919-a691-4100-8472-cab6a495f733\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:17:16.655825 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.655773    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv\" (UID: \"1d0189f7-fe9e-44a5-bcd5-24f125a267ff\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:17:16.656437 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.656414    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv\" (UID: \"1d0189f7-fe9e-44a5-bcd5-24f125a267ff\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:17:16.717120 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.717042    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx7nk\" (UniqueName: \"kubernetes.io/projected/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-kube-api-access-vx7nk\") pod \"payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv\" (UID: \"1d0189f7-fe9e-44a5-bcd5-24f125a267ff\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:17:16.739864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.739833    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:17:16.756664 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.756637    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfvjx\" (UniqueName: \"kubernetes.io/projected/a1a8c919-a691-4100-8472-cab6a495f733-kube-api-access-pfvjx\") pod \"payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq\" (UID: \"a1a8c919-a691-4100-8472-cab6a495f733\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:17:16.756790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.756744    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a8c919-a691-4100-8472-cab6a495f733-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq\" (UID: \"a1a8c919-a691-4100-8472-cab6a495f733\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:17:16.757641 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.757616    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a8c919-a691-4100-8472-cab6a495f733-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq\" (UID: \"a1a8c919-a691-4100-8472-cab6a495f733\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:17:16.826930 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.826888    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfvjx\" (UniqueName: \"kubernetes.io/projected/a1a8c919-a691-4100-8472-cab6a495f733-kube-api-access-pfvjx\") pod \"payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq\" (UID: \"a1a8c919-a691-4100-8472-cab6a495f733\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:17:16.891950 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:16.891910    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:17:17.130069 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:17.130045    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"]
Nov 04 02:17:17.130417 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:17:17.130396    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d0189f7_fe9e_44a5_bcd5_24f125a267ff.slice/crio-72a33662daa40a1a66e2ccd2b295431cdd18990a459b1f4353027ea5aea5195e WatchSource:0}: Error finding container 72a33662daa40a1a66e2ccd2b295431cdd18990a459b1f4353027ea5aea5195e: Status 404 returned error can't find the container with id 72a33662daa40a1a66e2ccd2b295431cdd18990a459b1f4353027ea5aea5195e
Nov 04 02:17:17.276096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:17.276064    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"]
Nov 04 02:17:17.277157 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:17:17.277128    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a8c919_a691_4100_8472_cab6a495f733.slice/crio-0ebf53b6dc3582d982025340b45375cfc652240282b274a3b7d5c6a6bf60f924 WatchSource:0}: Error finding container 0ebf53b6dc3582d982025340b45375cfc652240282b274a3b7d5c6a6bf60f924: Status 404 returned error can't find the container with id 0ebf53b6dc3582d982025340b45375cfc652240282b274a3b7d5c6a6bf60f924
Nov 04 02:17:18.069298 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:18.069263    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq" event={"ID":"a1a8c919-a691-4100-8472-cab6a495f733","Type":"ContainerStarted","Data":"3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1"}
Nov 04 02:17:18.069534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:18.069321    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:17:18.069534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:18.069347    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq" event={"ID":"a1a8c919-a691-4100-8472-cab6a495f733","Type":"ContainerStarted","Data":"0ebf53b6dc3582d982025340b45375cfc652240282b274a3b7d5c6a6bf60f924"}
Nov 04 02:17:18.070994 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:18.070970    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv" event={"ID":"1d0189f7-fe9e-44a5-bcd5-24f125a267ff","Type":"ContainerStarted","Data":"c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496"}
Nov 04 02:17:18.071092 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:18.071010    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv" event={"ID":"1d0189f7-fe9e-44a5-bcd5-24f125a267ff","Type":"ContainerStarted","Data":"72a33662daa40a1a66e2ccd2b295431cdd18990a459b1f4353027ea5aea5195e"}
Nov 04 02:17:18.071092 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:18.071039    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:17:18.188714 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:18.188660    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq" podStartSLOduration=2.188646219 podStartE2EDuration="2.188646219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:18.184288573 +0000 UTC m=+1299.989745199" watchObservedRunningTime="2026-11-04 02:17:18.188646219 +0000 UTC m=+1299.994102900"
Nov 04 02:17:18.300266 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:18.300215    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv" podStartSLOduration=2.300194809 podStartE2EDuration="2.300194809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:18.299536189 +0000 UTC m=+1300.104992821" watchObservedRunningTime="2026-11-04 02:17:18.300194809 +0000 UTC m=+1300.105651483"
Nov 04 02:17:20.694924 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:20.694828    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:20.918656 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:20.918615    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:21.119255 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:21.119218    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:22.598224 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:22.598193    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:23.168960 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.168924    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"]
Nov 04 02:17:23.174365 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.174338    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.231017 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.230982    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"]
Nov 04 02:17:23.274463 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.274430    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:17:23.274463 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.274443    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:17:23.284470 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.284438    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fgln\" (UniqueName: \"kubernetes.io/projected/d06e156c-8dcc-4d4b-b731-24b4615291d9-kube-api-access-4fgln\") pod \"maas-api-revoke-sub-72bf9b8f-c6m9h\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.284621 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.284562    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-72bf9b8f-c6m9h\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.284621 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.284613    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-72bf9b8f-c6m9h\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.323104 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.323054    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"]
Nov 04 02:17:23.327365 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.327335    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.385739 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.385704    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-72bf9b8f-c6m9h\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.385934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.385751    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-72bf9b8f-c6m9h\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.386316 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.385953    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fgln\" (UniqueName: \"kubernetes.io/projected/d06e156c-8dcc-4d4b-b731-24b4615291d9-kube-api-access-4fgln\") pod \"maas-api-revoke-sub-72bf9b8f-c6m9h\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.386557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.386532    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-72bf9b8f-c6m9h\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.386670 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.386608    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"]
Nov 04 02:17:23.388922 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.388893    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-72bf9b8f-c6m9h\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.447288 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.447214    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fgln\" (UniqueName: \"kubernetes.io/projected/d06e156c-8dcc-4d4b-b731-24b4615291d9-kube-api-access-4fgln\") pod \"maas-api-revoke-sub-72bf9b8f-c6m9h\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.487500 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.487468    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-plugins-config-volume\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.487650 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.487579    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55l8t\" (UniqueName: \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-kube-api-access-55l8t\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.487844 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.487823    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-routing-overlay-volume\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.487912 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.487899    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-provider-credentials\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.491222 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.491204    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:23.591819 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.589175    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-provider-credentials\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.591819 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.589256    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-plugins-config-volume\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.591819 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.589305    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55l8t\" (UniqueName: \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-kube-api-access-55l8t\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.591819 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.589489    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-routing-overlay-volume\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.591819 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.590428    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-routing-overlay-volume\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.591819 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.591578    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-plugins-config-volume\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.599546 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.593231    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-provider-credentials\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.652636 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.652599    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55l8t\" (UniqueName: \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-kube-api-access-55l8t\") pod \"payload-processing-external-model-6ccf9f4fbd-mg95w\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") " pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:23.878096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.877664    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"]
Nov 04 02:17:23.944153 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:23.944122    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:24.119879 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:24.119842    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h" event={"ID":"d06e156c-8dcc-4d4b-b731-24b4615291d9","Type":"ContainerStarted","Data":"1b88f58ce94c4bec8967c2b95f433ad5610db94dd56f4720f8f63a94ee218e30"}
Nov 04 02:17:24.339279 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:24.339246    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"]
Nov 04 02:17:24.339690 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:17:24.339666    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2704e4da_d3aa_4c9c_bf29_c677270630c5.slice/crio-39f56361cbd75f0f5fe9ca223308051a66148b4c9a1a3a9bb3afaeafba3d5aa4 WatchSource:0}: Error finding container 39f56361cbd75f0f5fe9ca223308051a66148b4c9a1a3a9bb3afaeafba3d5aa4: Status 404 returned error can't find the container with id 39f56361cbd75f0f5fe9ca223308051a66148b4c9a1a3a9bb3afaeafba3d5aa4
Nov 04 02:17:25.130126 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:25.130034    4512 generic.go:356] "Generic (PLEG): container finished" podID="d06e156c-8dcc-4d4b-b731-24b4615291d9" containerID="eb11cbc7feab63b2d991e59f171ea33fa08bcdc2d6c8682c0016b87f36c30647" exitCode=0
Nov 04 02:17:25.130126 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:25.130090    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h" event={"ID":"d06e156c-8dcc-4d4b-b731-24b4615291d9","Type":"ContainerDied","Data":"eb11cbc7feab63b2d991e59f171ea33fa08bcdc2d6c8682c0016b87f36c30647"}
Nov 04 02:17:25.132063 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:25.132029    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" event={"ID":"2704e4da-d3aa-4c9c-bf29-c677270630c5","Type":"ContainerStarted","Data":"d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c"}
Nov 04 02:17:25.132187 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:25.132094    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:17:25.132187 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:25.132118    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" event={"ID":"2704e4da-d3aa-4c9c-bf29-c677270630c5","Type":"ContainerStarted","Data":"39f56361cbd75f0f5fe9ca223308051a66148b4c9a1a3a9bb3afaeafba3d5aa4"}
Nov 04 02:17:25.369579 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:25.369523    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podStartSLOduration=2.369485644 podStartE2EDuration="2.369485644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:25.368546866 +0000 UTC m=+1307.174003501" watchObservedRunningTime="2026-11-04 02:17:25.369485644 +0000 UTC m=+1307.174942277"
Nov 04 02:17:25.960591 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:25.960547    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:26.314719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:26.314688    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:26.318892 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:26.318868    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-xks-ca\") pod \"d06e156c-8dcc-4d4b-b731-24b4615291d9\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") "
Nov 04 02:17:26.319003 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:26.318992    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-openshift-ca\") pod \"d06e156c-8dcc-4d4b-b731-24b4615291d9\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") "
Nov 04 02:17:26.319081 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:26.319037    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d06e156c-8dcc-4d4b-b731-24b4615291d9-kube-api-access-4fgln\" (UniqueName: \"kubernetes.io/projected/d06e156c-8dcc-4d4b-b731-24b4615291d9-kube-api-access-4fgln\") pod \"d06e156c-8dcc-4d4b-b731-24b4615291d9\" (UID: \"d06e156c-8dcc-4d4b-b731-24b4615291d9\") "
Nov 04 02:17:26.319422 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:26.319391    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-openshift-ca" pod "d06e156c-8dcc-4d4b-b731-24b4615291d9" (UID: "d06e156c-8dcc-4d4b-b731-24b4615291d9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:17:26.319595 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:26.319570    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:26.321255 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:26.321232    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-xks-ca" pod "d06e156c-8dcc-4d4b-b731-24b4615291d9" (UID: "d06e156c-8dcc-4d4b-b731-24b4615291d9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:17:26.321636 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:26.321297    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d06e156c-8dcc-4d4b-b731-24b4615291d9-kube-api-access-4fgln" pod "d06e156c-8dcc-4d4b-b731-24b4615291d9" (UID: "d06e156c-8dcc-4d4b-b731-24b4615291d9"). InnerVolumeSpecName "kube-api-access-4fgln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:17:26.420906 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:26.420861    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d06e156c-8dcc-4d4b-b731-24b4615291d9-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:26.420906 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:26.420894    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fgln\" (UniqueName: \"kubernetes.io/projected/d06e156c-8dcc-4d4b-b731-24b4615291d9-kube-api-access-4fgln\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:26.778536 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:26.778460    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:17:27.147645 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:27.147565    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"
Nov 04 02:17:27.147795 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:27.147564    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h" event={"ID":"d06e156c-8dcc-4d4b-b731-24b4615291d9","Type":"ContainerDied","Data":"1b88f58ce94c4bec8967c2b95f433ad5610db94dd56f4720f8f63a94ee218e30"}
Nov 04 02:17:27.147795 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:27.147692    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b88f58ce94c4bec8967c2b95f433ad5610db94dd56f4720f8f63a94ee218e30"
Nov 04 02:17:27.718931 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:27.718899    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:29.082373 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:29.082344    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:17:29.082832 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:29.082410    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:17:30.789318 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:30.789282    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:31.577106 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:31.577065    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:33.863934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:33.863884    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:34.063845 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:34.063806    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:35.688354 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:35.688312    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:36.142179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:36.142140    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:17:39.979696 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:39.979647    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"]
Nov 04 02:17:39.980519 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:39.980486    4512 cpu_manager.go:414] "containerMap: removing container" podUID="d06e156c-8dcc-4d4b-b731-24b4615291d9" containerName="revoke-keys"
Nov 04 02:17:39.980577 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:39.980524    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d06e156c-8dcc-4d4b-b731-24b4615291d9" containerName="revoke-keys"
Nov 04 02:17:39.980682 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:39.980671    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="d06e156c-8dcc-4d4b-b731-24b4615291d9" containerName="revoke-keys"
Nov 04 02:17:39.986167 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:39.986150    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.040973 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.040943    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"]
Nov 04 02:17:40.084474 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.084452    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:17:40.084474 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.084466    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:17:40.097470 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.097443    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j96x\" (UniqueName: \"kubernetes.io/projected/c32c82c1-f62b-4818-862d-91f701414623-kube-api-access-2j96x\") pod \"maas-api-revoke-sub-767639c7-zlt8w\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.097592 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.097488    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c32c82c1-f62b-4818-862d-91f701414623-maas-api-xks-ca\") pod \"maas-api-revoke-sub-767639c7-zlt8w\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.097683 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.097669    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c32c82c1-f62b-4818-862d-91f701414623-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-767639c7-zlt8w\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.199100 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.199071    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c32c82c1-f62b-4818-862d-91f701414623-maas-api-xks-ca\") pod \"maas-api-revoke-sub-767639c7-zlt8w\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.199212 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.199190    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c32c82c1-f62b-4818-862d-91f701414623-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-767639c7-zlt8w\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.199251 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.199240    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j96x\" (UniqueName: \"kubernetes.io/projected/c32c82c1-f62b-4818-862d-91f701414623-kube-api-access-2j96x\") pod \"maas-api-revoke-sub-767639c7-zlt8w\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.199941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.199887    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c32c82c1-f62b-4818-862d-91f701414623-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-767639c7-zlt8w\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.201556 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.201527    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c32c82c1-f62b-4818-862d-91f701414623-maas-api-xks-ca\") pod \"maas-api-revoke-sub-767639c7-zlt8w\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.234594 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.234562    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:40.260286 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.260245    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j96x\" (UniqueName: \"kubernetes.io/projected/c32c82c1-f62b-4818-862d-91f701414623-kube-api-access-2j96x\") pod \"maas-api-revoke-sub-767639c7-zlt8w\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.304194 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.304170    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:40.587536 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.587420    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:40.687727 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:40.687700    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"]
Nov 04 02:17:40.688067 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:17:40.688045    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc32c82c1_f62b_4818_862d_91f701414623.slice/crio-028448c137523d884f48a520e0101d94ec5db3b91835cd4631de90cd3d26d5d2 WatchSource:0}: Error finding container 028448c137523d884f48a520e0101d94ec5db3b91835cd4631de90cd3d26d5d2: Status 404 returned error can't find the container with id 028448c137523d884f48a520e0101d94ec5db3b91835cd4631de90cd3d26d5d2
Nov 04 02:17:41.272096 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:41.272061    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w" event={"ID":"c32c82c1-f62b-4818-862d-91f701414623","Type":"ContainerStarted","Data":"34c054b3cde48a6bcda416e5c31f77f1587a551a690213678ed00b2beb05d314"}
Nov 04 02:17:41.272464 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:41.272127    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w" event={"ID":"c32c82c1-f62b-4818-862d-91f701414623","Type":"ContainerStarted","Data":"028448c137523d884f48a520e0101d94ec5db3b91835cd4631de90cd3d26d5d2"}
Nov 04 02:17:41.389669 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:41.389616    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w" podStartSLOduration=2.389599726 podStartE2EDuration="2.389599726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:41.386940788 +0000 UTC m=+1323.192397425" watchObservedRunningTime="2026-11-04 02:17:41.389599726 +0000 UTC m=+1323.195056358"
Nov 04 02:17:42.280989 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:42.280950    4512 generic.go:356] "Generic (PLEG): container finished" podID="c32c82c1-f62b-4818-862d-91f701414623" containerID="34c054b3cde48a6bcda416e5c31f77f1587a551a690213678ed00b2beb05d314" exitCode=0
Nov 04 02:17:42.281405 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:42.281035    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w" event={"ID":"c32c82c1-f62b-4818-862d-91f701414623","Type":"ContainerDied","Data":"34c054b3cde48a6bcda416e5c31f77f1587a551a690213678ed00b2beb05d314"}
Nov 04 02:17:43.213115 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.213082    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"]
Nov 04 02:17:43.328779 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.328747    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"]
Nov 04 02:17:43.480289 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.480225    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:43.657197 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.655974    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c32c82c1-f62b-4818-862d-91f701414623-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c32c82c1-f62b-4818-862d-91f701414623-maas-api-xks-ca\") pod \"c32c82c1-f62b-4818-862d-91f701414623\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") "
Nov 04 02:17:43.657197 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.656223    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c32c82c1-f62b-4818-862d-91f701414623-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c32c82c1-f62b-4818-862d-91f701414623-maas-api-openshift-ca\") pod \"c32c82c1-f62b-4818-862d-91f701414623\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") "
Nov 04 02:17:43.657197 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.656273    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c32c82c1-f62b-4818-862d-91f701414623-kube-api-access-2j96x\" (UniqueName: \"kubernetes.io/projected/c32c82c1-f62b-4818-862d-91f701414623-kube-api-access-2j96x\") pod \"c32c82c1-f62b-4818-862d-91f701414623\" (UID: \"c32c82c1-f62b-4818-862d-91f701414623\") "
Nov 04 02:17:43.657619 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.657592    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:43.658805 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.658170    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c32c82c1-f62b-4818-862d-91f701414623-maas-api-openshift-ca" pod "c32c82c1-f62b-4818-862d-91f701414623" (UID: "c32c82c1-f62b-4818-862d-91f701414623"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:17:43.676053 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.675985    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c32c82c1-f62b-4818-862d-91f701414623-maas-api-xks-ca" pod "c32c82c1-f62b-4818-862d-91f701414623" (UID: "c32c82c1-f62b-4818-862d-91f701414623"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:17:43.689242 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.689203    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c32c82c1-f62b-4818-862d-91f701414623-kube-api-access-2j96x" pod "c32c82c1-f62b-4818-862d-91f701414623" (UID: "c32c82c1-f62b-4818-862d-91f701414623"). InnerVolumeSpecName "kube-api-access-2j96x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:17:43.757619 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.757589    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c32c82c1-f62b-4818-862d-91f701414623-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.757619 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.757615    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j96x\" (UniqueName: \"kubernetes.io/projected/c32c82c1-f62b-4818-862d-91f701414623-kube-api-access-2j96x\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.757778 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.757625    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c32c82c1-f62b-4818-862d-91f701414623-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:43.857457 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:43.857415    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:44.301953 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:44.301919    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"
Nov 04 02:17:44.302124 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:44.301950    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w" event={"ID":"c32c82c1-f62b-4818-862d-91f701414623","Type":"ContainerDied","Data":"028448c137523d884f48a520e0101d94ec5db3b91835cd4631de90cd3d26d5d2"}
Nov 04 02:17:44.302124 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:44.302001    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="028448c137523d884f48a520e0101d94ec5db3b91835cd4631de90cd3d26d5d2"
Nov 04 02:17:45.705556 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:45.705492    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:46.140875 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:46.140835    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:17:46.386444 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:46.386413    4512 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-26d257c38a93: secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:46.386444 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:46.386441    4512 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4: secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:46.386716 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:46.386549    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials podName:1d3300f6-b801-481d-a7f9-b76114860e85 nodeName:}" failed. No retries permitted until 2026-11-04 02:17:46.886498959 +0000 UTC m=+1328.691955586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials") pod "payload-processing-external-model-6945d56d5f-5trg4" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85") : secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:46.807236 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:46.807199    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:46.891791 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:46.891761    4512 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-26d257c38a93: secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:46.891791 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:46.891788    4512 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4: secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:46.891963 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:46.891846    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials podName:1d3300f6-b801-481d-a7f9-b76114860e85 nodeName:}" failed. No retries permitted until 2026-11-04 02:17:47.89183055 +0000 UTC m=+1329.697287174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials") pod "payload-processing-external-model-6945d56d5f-5trg4" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85") : secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:47.901943 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:47.901907    4512 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-26d257c38a93: secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:47.901943 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:47.901944    4512 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4: secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:47.902397 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:47.902028    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials podName:1d3300f6-b801-481d-a7f9-b76114860e85 nodeName:}" failed. No retries permitted until 2026-11-04 02:17:49.902006575 +0000 UTC m=+1331.707463208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials") pod "payload-processing-external-model-6945d56d5f-5trg4" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85") : secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:48.149025 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.148989    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:48.214227 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.214158    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24" podUID="cd99bf0f-215a-43b1-921d-cf1fc6cb4adb" containerName="payload-processing" containerID="cri-o://a628b9beba2fc0b9866fc9707948c78de090d4c44a71e3608c26cf957491d432" gracePeriod=25
Nov 04 02:17:48.329584 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.329551    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd" podUID="6b45e578-cd49-44ad-8b21-b2e82147c4a6" containerName="payload-pre-processing" containerID="cri-o://47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683" gracePeriod=25
Nov 04 02:17:48.343261 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.343231    4512 generic.go:356] "Generic (PLEG): container finished" podID="cd99bf0f-215a-43b1-921d-cf1fc6cb4adb" containerID="a628b9beba2fc0b9866fc9707948c78de090d4c44a71e3608c26cf957491d432" exitCode=0
Nov 04 02:17:48.343369 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.343302    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24" event={"ID":"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb","Type":"ContainerDied","Data":"a628b9beba2fc0b9866fc9707948c78de090d4c44a71e3608c26cf957491d432"}
Nov 04 02:17:48.345248 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.345224    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:48.510242 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.510216    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:17:48.608813 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.608765    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-kube-api-access-vwb4r\" (UniqueName: \"kubernetes.io/projected/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-kube-api-access-vwb4r\") pod \"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb\" (UID: \"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb\") "
Nov 04 02:17:48.608953 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.608929    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-plugins-config-volume\") pod \"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb\" (UID: \"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb\") "
Nov 04 02:17:48.609332 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.609304    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-plugins-config-volume" pod "cd99bf0f-215a-43b1-921d-cf1fc6cb4adb" (UID: "cd99bf0f-215a-43b1-921d-cf1fc6cb4adb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:17:48.609461 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.609428    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:48.611350 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.611324    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-kube-api-access-vwb4r" pod "cd99bf0f-215a-43b1-921d-cf1fc6cb4adb" (UID: "cd99bf0f-215a-43b1-921d-cf1fc6cb4adb"). InnerVolumeSpecName "kube-api-access-vwb4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:17:48.617580 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.617558    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:17:48.709914 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.709885    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b45e578-cd49-44ad-8b21-b2e82147c4a6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b45e578-cd49-44ad-8b21-b2e82147c4a6-plugins-config-volume\") pod \"6b45e578-cd49-44ad-8b21-b2e82147c4a6\" (UID: \"6b45e578-cd49-44ad-8b21-b2e82147c4a6\") "
Nov 04 02:17:48.710102 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.710085    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b45e578-cd49-44ad-8b21-b2e82147c4a6-kube-api-access-rmb2v\" (UniqueName: \"kubernetes.io/projected/6b45e578-cd49-44ad-8b21-b2e82147c4a6-kube-api-access-rmb2v\") pod \"6b45e578-cd49-44ad-8b21-b2e82147c4a6\" (UID: \"6b45e578-cd49-44ad-8b21-b2e82147c4a6\") "
Nov 04 02:17:48.710260 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.710234    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b45e578-cd49-44ad-8b21-b2e82147c4a6-plugins-config-volume" pod "6b45e578-cd49-44ad-8b21-b2e82147c4a6" (UID: "6b45e578-cd49-44ad-8b21-b2e82147c4a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:17:48.710556 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.710537    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b45e578-cd49-44ad-8b21-b2e82147c4a6-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:48.710625 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.710560    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwb4r\" (UniqueName: \"kubernetes.io/projected/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb-kube-api-access-vwb4r\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:48.712309 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.712284    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b45e578-cd49-44ad-8b21-b2e82147c4a6-kube-api-access-rmb2v" pod "6b45e578-cd49-44ad-8b21-b2e82147c4a6" (UID: "6b45e578-cd49-44ad-8b21-b2e82147c4a6"). InnerVolumeSpecName "kube-api-access-rmb2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:17:48.812387 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:48.812354    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmb2v\" (UniqueName: \"kubernetes.io/projected/6b45e578-cd49-44ad-8b21-b2e82147c4a6-kube-api-access-rmb2v\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:17:49.351685 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.351654    4512 generic.go:356] "Generic (PLEG): container finished" podID="6b45e578-cd49-44ad-8b21-b2e82147c4a6" containerID="47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683" exitCode=0
Nov 04 02:17:49.352101 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.351710    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"
Nov 04 02:17:49.352101 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.351732    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd" event={"ID":"6b45e578-cd49-44ad-8b21-b2e82147c4a6","Type":"ContainerDied","Data":"47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683"}
Nov 04 02:17:49.352101 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.351788    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd" event={"ID":"6b45e578-cd49-44ad-8b21-b2e82147c4a6","Type":"ContainerDied","Data":"3370d4eeedbb8595dc8941700bbff3017eba675b3105ca03add0929e6c12dbf9"}
Nov 04 02:17:49.352101 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.351819    4512 scope.go:122] "RemoveContainer" containerID="47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683"
Nov 04 02:17:49.353581 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.353562    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24" event={"ID":"cd99bf0f-215a-43b1-921d-cf1fc6cb4adb","Type":"ContainerDied","Data":"d965bed02fced0c7cd66131d84984f2163026effc282957c1892bd1444b75d11"}
Nov 04 02:17:49.353701 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.353678    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"
Nov 04 02:17:49.364203 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.364183    4512 scope.go:122] "RemoveContainer" containerID="47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683"
Nov 04 02:17:49.364481 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:49.364464    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683\": container with ID starting with 47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683 not found: ID does not exist" containerID="47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683"
Nov 04 02:17:49.364583 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.364489    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683"} err="failed to get container status \"47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683\": rpc error: code = NotFound desc = could not find container \"47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683\": container with ID starting with 47debda9d30144b164651c688fa9883d6a9046c07fa9e1b1811194aaaa45d683 not found: ID does not exist"
Nov 04 02:17:49.364583 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.364524    4512 scope.go:122] "RemoveContainer" containerID="a628b9beba2fc0b9866fc9707948c78de090d4c44a71e3608c26cf957491d432"
Nov 04 02:17:49.519080 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.519050    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"]
Nov 04 02:17:49.522678 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.522649    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd"]
Nov 04 02:17:49.630199 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.630171    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"]
Nov 04 02:17:49.633933 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:49.633913    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24"]
Nov 04 02:17:49.923182 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:49.923105    4512 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-26d257c38a93: secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:49.923182 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:49.923131    4512 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4: secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:49.923365 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:49.923186    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials podName:1d3300f6-b801-481d-a7f9-b76114860e85 nodeName:}" failed. No retries permitted until 2026-11-04 02:17:53.92317106 +0000 UTC m=+1335.728627674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials") pod "payload-processing-external-model-6945d56d5f-5trg4" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85") : secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:50.314018 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:50.313991    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b45e578-cd49-44ad-8b21-b2e82147c4a6" path="/var/lib/kubelet/pods/6b45e578-cd49-44ad-8b21-b2e82147c4a6/volumes"
Nov 04 02:17:50.314398 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:50.314383    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd99bf0f-215a-43b1-921d-cf1fc6cb4adb" path="/var/lib/kubelet/pods/cd99bf0f-215a-43b1-921d-cf1fc6cb4adb/volumes"
Nov 04 02:17:50.572311 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:50.572217    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:52.662081 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:52.662041    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:53.877477 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.877441    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"]
Nov 04 02:17:53.878093 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.878079    4512 cpu_manager.go:414] "containerMap: removing container" podUID="cd99bf0f-215a-43b1-921d-cf1fc6cb4adb" containerName="payload-processing"
Nov 04 02:17:53.878145 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.878094    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd99bf0f-215a-43b1-921d-cf1fc6cb4adb" containerName="payload-processing"
Nov 04 02:17:53.878145 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.878119    4512 cpu_manager.go:414] "containerMap: removing container" podUID="6b45e578-cd49-44ad-8b21-b2e82147c4a6" containerName="payload-pre-processing"
Nov 04 02:17:53.878145 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.878124    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b45e578-cd49-44ad-8b21-b2e82147c4a6" containerName="payload-pre-processing"
Nov 04 02:17:53.878238 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.878154    4512 cpu_manager.go:414] "containerMap: removing container" podUID="c32c82c1-f62b-4818-862d-91f701414623" containerName="revoke-keys"
Nov 04 02:17:53.878238 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.878159    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c32c82c1-f62b-4818-862d-91f701414623" containerName="revoke-keys"
Nov 04 02:17:53.878310 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.878252    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="c32c82c1-f62b-4818-862d-91f701414623" containerName="revoke-keys"
Nov 04 02:17:53.878310 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.878262    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd99bf0f-215a-43b1-921d-cf1fc6cb4adb" containerName="payload-processing"
Nov 04 02:17:53.878310 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.878272    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b45e578-cd49-44ad-8b21-b2e82147c4a6" containerName="payload-pre-processing"
Nov 04 02:17:53.883140 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.883122    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:53.929140 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.929114    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"]
Nov 04 02:17:53.934712 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.934692    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:53.975299 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975273    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:53.975400 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975323    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:53.975400 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975341    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:53.975500 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:53.975478    4512 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-26d257c38a93: secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:53.975500 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975488    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:53.975576 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:53.975522    4512 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4: secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:53.975576 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975552    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:53.975642 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:53.975581    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials podName:1d3300f6-b801-481d-a7f9-b76114860e85 nodeName:}" failed. No retries permitted until 2026-11-04 02:18:01.97556159 +0000 UTC m=+1343.781018200 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials") pod "payload-processing-external-model-6945d56d5f-5trg4" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85") : secret "agc-cred-26d257c38a93" not found
Nov 04 02:17:53.975685 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975637    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:53.975719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975681    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:53.975719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975704    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:53.975781 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975729    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:53.975828 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975808    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:53.975866 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975848    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:53.975926 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975910    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:53.975962 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975950    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:53.975992 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.975966    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl275\" (UniqueName: \"kubernetes.io/projected/5c820e07-bdfa-43ee-bce8-4d38241f414a-kube-api-access-tl275\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:53.976055 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.976039    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:53.976105 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.976067    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c820e07-bdfa-43ee-bce8-4d38241f414a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:53.976105 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.976085    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:53.976200 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.976121    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg4r7\" (UniqueName: \"kubernetes.io/projected/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-kube-api-access-sg4r7\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:53.982819 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:53.982799    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"]
Nov 04 02:17:54.054567 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.054529    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"]
Nov 04 02:17:54.077407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077375    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.077407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077407    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl275\" (UniqueName: \"kubernetes.io/projected/5c820e07-bdfa-43ee-bce8-4d38241f414a-kube-api-access-tl275\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.077688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077433    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.077688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077456    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c820e07-bdfa-43ee-bce8-4d38241f414a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.077688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077485    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.077688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077547    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg4r7\" (UniqueName: \"kubernetes.io/projected/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-kube-api-access-sg4r7\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.077688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077618    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.077688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077672    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.078041 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077695    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.078041 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077728    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.078041 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077845    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.078041 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.077856    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.078260 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078116    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.078260 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078158    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.078260 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078200    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.078260 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078230    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.078469 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078275    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.078469 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078343    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.078469 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078406    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.078469 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078455    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.078707 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078488    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.078707 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078607    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c820e07-bdfa-43ee-bce8-4d38241f414a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.078844 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.078822    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.079091 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.079070    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.079207 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.079182    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.079554 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.079527    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.079678 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.079584    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.079858 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.079836    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.081080 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.081051    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.081572 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.081547    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.081883 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.081856    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.082022 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.082000    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.139111 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.139027    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c820e07-bdfa-43ee-bce8-4d38241f414a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.139341 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.139314    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg4r7\" (UniqueName: \"kubernetes.io/projected/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-kube-api-access-sg4r7\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.139456 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.139314    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl275\" (UniqueName: \"kubernetes.io/projected/5c820e07-bdfa-43ee-bce8-4d38241f414a-kube-api-access-tl275\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-tnrml\" (UID: \"5c820e07-bdfa-43ee-bce8-4d38241f414a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.140155 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.140119    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/907abcc8-9f0d-4809-b0d4-56326e5fe3d8-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5\" (UID: \"907abcc8-9f0d-4809-b0d4-56326e5fe3d8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.198935 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.198904    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:54.249287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.249262    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:54.634948 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.634919    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"]
Nov 04 02:17:54.635335 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:17:54.635307    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c820e07_bdfa_43ee_bce8_4d38241f414a.slice/crio-76fb2b1c7b664052fe870b37c52ee13e6c8420c91e32caa6835876e8b95110a5 WatchSource:0}: Error finding container 76fb2b1c7b664052fe870b37c52ee13e6c8420c91e32caa6835876e8b95110a5: Status 404 returned error can't find the container with id 76fb2b1c7b664052fe870b37c52ee13e6c8420c91e32caa6835876e8b95110a5
Nov 04 02:17:54.638017 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.637982    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:17:54.638125 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.638043    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:17:54.638125 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.638074    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:17:54.689665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.689633    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"]
Nov 04 02:17:54.703775 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.703546    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:17:54.703775 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.703633    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:17:54.703775 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:54.703663    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:17:55.403654 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:55.403616    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" event={"ID":"5c820e07-bdfa-43ee-bce8-4d38241f414a","Type":"ContainerStarted","Data":"2b9e83b882f268f609191cc52668fc0103e14355a6ddefe81058411dc55a742f"}
Nov 04 02:17:55.404133 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:55.403684    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" event={"ID":"5c820e07-bdfa-43ee-bce8-4d38241f414a","Type":"ContainerStarted","Data":"76fb2b1c7b664052fe870b37c52ee13e6c8420c91e32caa6835876e8b95110a5"}
Nov 04 02:17:55.405644 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:55.405615    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" event={"ID":"907abcc8-9f0d-4809-b0d4-56326e5fe3d8","Type":"ContainerStarted","Data":"45116071c74cede5c18de7c5c25108da9524da6944045f56ecb7507a92ea1b07"}
Nov 04 02:17:55.405772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:55.405665    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" event={"ID":"907abcc8-9f0d-4809-b0d4-56326e5fe3d8","Type":"ContainerStarted","Data":"406a9a5de2978b7e16b2e09e3adbe076f240a4c68c7db5273c59ca104c96ea63"}
Nov 04 02:17:55.530014 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:55.529950    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" podStartSLOduration=2.5299353399999998 podStartE2EDuration="2.52993534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:55.528006627 +0000 UTC m=+1337.333463260" watchObservedRunningTime="2026-11-04 02:17:55.52993534 +0000 UTC m=+1337.335391972"
Nov 04 02:17:55.577820 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:55.577782    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:55.649477 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:55.649287    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" podStartSLOduration=2.649272269 podStartE2EDuration="2.649272269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:55.647634173 +0000 UTC m=+1337.453090804" watchObservedRunningTime="2026-11-04 02:17:55.649272269 +0000 UTC m=+1337.454728901"
Nov 04 02:17:56.149694 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:56.149651    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:17:56.201531 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:56.200340    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:17:56.225535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:56.223870    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 04 02:17:56.225535 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:56.223932    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" podUID="5c820e07-bdfa-43ee-bce8-4d38241f414a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 04 02:17:56.250533 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:56.250461    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:17:56.252298 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:56.252261    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 04 02:17:56.252449 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:56.252323    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" podUID="907abcc8-9f0d-4809-b0d4-56326e5fe3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 04 02:17:56.785855 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:17:56.785816    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:17:57.202988 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:57.202892    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 04 02:17:57.203281 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:57.203239    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" podUID="5c820e07-bdfa-43ee-bce8-4d38241f414a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 04 02:17:57.250558 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:57.249938    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 04 02:17:57.250558 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:57.250009    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" podUID="907abcc8-9f0d-4809-b0d4-56326e5fe3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 04 02:17:57.269215 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:57.269184    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:58.200001 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:58.199962    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 04 02:17:58.200767 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:58.200735    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" podUID="5c820e07-bdfa-43ee-bce8-4d38241f414a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 04 02:17:58.255164 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:58.255130    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 04 02:17:58.255436 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:58.255409    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" podUID="907abcc8-9f0d-4809-b0d4-56326e5fe3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 04 02:17:59.215534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:59.210617    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 04 02:17:59.215534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:59.210727    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" podUID="5c820e07-bdfa-43ee-bce8-4d38241f414a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 04 02:17:59.249665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:59.249632    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 04 02:17:59.249913 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:17:59.249883    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" podUID="907abcc8-9f0d-4809-b0d4-56326e5fe3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 04 02:18:00.204532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:00.203911    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 04 02:18:00.204532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:00.203979    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" podUID="5c820e07-bdfa-43ee-bce8-4d38241f414a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 04 02:18:00.249951 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:00.249911    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 04 02:18:00.250568 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:00.250535    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" podUID="907abcc8-9f0d-4809-b0d4-56326e5fe3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 04 02:18:01.202054 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:01.202017    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 04 02:18:01.202384 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:01.202356    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" podUID="5c820e07-bdfa-43ee-bce8-4d38241f414a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 04 02:18:01.251046 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:01.251007    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 04 02:18:01.251687 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:01.251652    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" podUID="907abcc8-9f0d-4809-b0d4-56326e5fe3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 04 02:18:01.369783 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:01.369733    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:01.625533 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:01.621609    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:01.837198 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:01.837159    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:02.026529 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:02.025745    4512 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-26d257c38a93: secret "agc-cred-26d257c38a93" not found
Nov 04 02:18:02.026529 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:02.025788    4512 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4: secret "agc-cred-26d257c38a93" not found
Nov 04 02:18:02.026529 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:02.025854    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials podName:1d3300f6-b801-481d-a7f9-b76114860e85 nodeName:}" failed. No retries permitted until 2026-11-04 02:18:18.025834328 +0000 UTC m=+1359.831290944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials") pod "payload-processing-external-model-6945d56d5f-5trg4" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85") : secret "agc-cred-26d257c38a93" not found
Nov 04 02:18:02.200531 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:02.199938    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 04 02:18:02.200531 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:02.200005    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" podUID="5c820e07-bdfa-43ee-bce8-4d38241f414a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 04 02:18:02.251528 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:02.250962    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 04 02:18:02.251528 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:02.251046    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" podUID="907abcc8-9f0d-4809-b0d4-56326e5fe3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 04 02:18:03.199884 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:03.199841    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 04 02:18:03.200067 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:03.199912    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml" podUID="5c820e07-bdfa-43ee-bce8-4d38241f414a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 04 02:18:03.249710 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:03.249669    4512 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Nov 04 02:18:03.249903 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:03.249740    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5" podUID="907abcc8-9f0d-4809-b0d4-56326e5fe3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Nov 04 02:18:04.813358 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.813324    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:18:04.814228 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.814207    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:18:04.814981 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.814957    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml"
Nov 04 02:18:04.834805 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.834770    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:18:04.835303 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.835272    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:18:04.836565 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.836540    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5"
Nov 04 02:18:04.862143 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.862095    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"]
Nov 04 02:18:04.867790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.867767    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:04.883123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.882741    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0c819d14-g2tf8\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:04.883123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.882851    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0c819d14-g2tf8\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:04.883123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.882879    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2plb\" (UniqueName: \"kubernetes.io/projected/3c26fc7d-912d-4d7e-8a21-b7b26942e361-kube-api-access-x2plb\") pod \"maas-api-revoke-sub-0c819d14-g2tf8\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:04.920294 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.920255    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:18:04.920472 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.920416    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:18:04.924211 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.924184    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"]
Nov 04 02:18:04.983564 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.983535    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2plb\" (UniqueName: \"kubernetes.io/projected/3c26fc7d-912d-4d7e-8a21-b7b26942e361-kube-api-access-x2plb\") pod \"maas-api-revoke-sub-0c819d14-g2tf8\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:04.983773 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.983721    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0c819d14-g2tf8\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:04.983843 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.983775    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0c819d14-g2tf8\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:04.984409 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.984380    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0c819d14-g2tf8\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:04.986335 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:04.986314    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0c819d14-g2tf8\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:05.047376 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:05.047344    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2plb\" (UniqueName: \"kubernetes.io/projected/3c26fc7d-912d-4d7e-8a21-b7b26942e361-kube-api-access-x2plb\") pod \"maas-api-revoke-sub-0c819d14-g2tf8\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:05.183195 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:05.183110    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:05.273726 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:05.273693    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:05.541802 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:05.541766    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:05.599082 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:05.599045    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"]
Nov 04 02:18:05.600020 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:18:05.599979    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c26fc7d_912d_4d7e_8a21_b7b26942e361.slice/crio-50cd57558231329b20c04fa1651faf2541458d3f36af2706d81ac8cb44535cd0 WatchSource:0}: Error finding container 50cd57558231329b20c04fa1651faf2541458d3f36af2706d81ac8cb44535cd0: Status 404 returned error can't find the container with id 50cd57558231329b20c04fa1651faf2541458d3f36af2706d81ac8cb44535cd0
Nov 04 02:18:05.751023 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:05.750986    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:05.952682 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:05.952651    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:06.140301 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:06.140217    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:18:06.154845 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:06.154813    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:06.578333 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:06.578300    4512 generic.go:356] "Generic (PLEG): container finished" podID="3c26fc7d-912d-4d7e-8a21-b7b26942e361" containerID="ed8def9a862cce9bf616ba358d62ea9fcac90319b3d62902a121dc23ebc87528" exitCode=0
Nov 04 02:18:06.578521 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:06.578379    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8" event={"ID":"3c26fc7d-912d-4d7e-8a21-b7b26942e361","Type":"ContainerDied","Data":"ed8def9a862cce9bf616ba358d62ea9fcac90319b3d62902a121dc23ebc87528"}
Nov 04 02:18:06.578521 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:06.578432    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8" event={"ID":"3c26fc7d-912d-4d7e-8a21-b7b26942e361","Type":"ContainerStarted","Data":"50cd57558231329b20c04fa1651faf2541458d3f36af2706d81ac8cb44535cd0"}
Nov 04 02:18:07.698684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.698494    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:07.776887 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.776860    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:07.848140 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.848041    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c26fc7d-912d-4d7e-8a21-b7b26942e361-kube-api-access-x2plb\" (UniqueName: \"kubernetes.io/projected/3c26fc7d-912d-4d7e-8a21-b7b26942e361-kube-api-access-x2plb\") pod \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") "
Nov 04 02:18:07.848140 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.848111    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-openshift-ca\") pod \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") "
Nov 04 02:18:07.848366 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.848293    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-xks-ca\") pod \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\" (UID: \"3c26fc7d-912d-4d7e-8a21-b7b26942e361\") "
Nov 04 02:18:07.848563 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.848500    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-openshift-ca" pod "3c26fc7d-912d-4d7e-8a21-b7b26942e361" (UID: "3c26fc7d-912d-4d7e-8a21-b7b26942e361"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:07.848769 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.848752    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:07.850413 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.850387    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-xks-ca" pod "3c26fc7d-912d-4d7e-8a21-b7b26942e361" (UID: "3c26fc7d-912d-4d7e-8a21-b7b26942e361"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:18:07.850553 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.850528    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c26fc7d-912d-4d7e-8a21-b7b26942e361-kube-api-access-x2plb" pod "3c26fc7d-912d-4d7e-8a21-b7b26942e361" (UID: "3c26fc7d-912d-4d7e-8a21-b7b26942e361"). InnerVolumeSpecName "kube-api-access-x2plb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:07.950174 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.950135    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c26fc7d-912d-4d7e-8a21-b7b26942e361-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:07.950174 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:07.950166    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2plb\" (UniqueName: \"kubernetes.io/projected/3c26fc7d-912d-4d7e-8a21-b7b26942e361-kube-api-access-x2plb\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:08.313458 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:08.313423    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:08.514072 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:08.514038    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:08.597412 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:08.597330    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"
Nov 04 02:18:08.597602 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:08.597329    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8" event={"ID":"3c26fc7d-912d-4d7e-8a21-b7b26942e361","Type":"ContainerDied","Data":"50cd57558231329b20c04fa1651faf2541458d3f36af2706d81ac8cb44535cd0"}
Nov 04 02:18:08.597602 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:08.597470    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50cd57558231329b20c04fa1651faf2541458d3f36af2706d81ac8cb44535cd0"
Nov 04 02:18:09.184283 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:09.184246    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:09.411321 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:09.411281    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:09.614238 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:09.614201    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:09.954759 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:09.954657    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:10.442989 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:10.442957    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"]
Nov 04 02:18:10.443417 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:10.443163    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerName="istio-proxy" containerID="cri-o://302c7287f172a1dfedfe4a9d40bfcf70e8781f04747895b08f01b0903a9971ed" gracePeriod=30
Nov 04 02:18:10.503246 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:10.503202    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:11.658132 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:11.658083    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:11.950560 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:11.950450    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:12.975103 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:12.975067    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:13.912585 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:13.912549    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:14.132407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:14.132366    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:14.371283 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:14.371254    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:15.679951 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.679900    4512 generic.go:356] "Generic (PLEG): container finished" podID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerID="302c7287f172a1dfedfe4a9d40bfcf70e8781f04747895b08f01b0903a9971ed" exitCode=0
Nov 04 02:18:15.680726 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.680696    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" event={"ID":"052e4f45-bd16-4ef9-9c17-10a331df3ad4","Type":"ContainerDied","Data":"302c7287f172a1dfedfe4a9d40bfcf70e8781f04747895b08f01b0903a9971ed"}
Nov 04 02:18:15.737425 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.737400    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:18:15.870340 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.870287    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-socket\") pod \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") "
Nov 04 02:18:15.870539 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.870372    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-credential-socket\") pod \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") "
Nov 04 02:18:15.870539 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.870470    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-podinfo\") pod \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") "
Nov 04 02:18:15.870675 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.870553    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-token\" (UniqueName: \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-token\") pod \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") "
Nov 04 02:18:15.870675 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.870614    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-kube-api-access-w64rb\" (UniqueName: \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-kube-api-access-w64rb\") pod \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") "
Nov 04 02:18:15.870675 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.870631    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istiod-ca-cert\") pod \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") "
Nov 04 02:18:15.870840 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.870680    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-envoy\") pod \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") "
Nov 04 02:18:15.870840 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.870731    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-data\") pod \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") "
Nov 04 02:18:15.870840 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.870796    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-certs\") pod \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\" (UID: \"052e4f45-bd16-4ef9-9c17-10a331df3ad4\") "
Nov 04 02:18:15.871693 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.871336    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-socket" pod "052e4f45-bd16-4ef9-9c17-10a331df3ad4" (UID: "052e4f45-bd16-4ef9-9c17-10a331df3ad4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.871693 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.871585    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-data" pod "052e4f45-bd16-4ef9-9c17-10a331df3ad4" (UID: "052e4f45-bd16-4ef9-9c17-10a331df3ad4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.871693 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.871628    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-certs" pod "052e4f45-bd16-4ef9-9c17-10a331df3ad4" (UID: "052e4f45-bd16-4ef9-9c17-10a331df3ad4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.871693 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.871654    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-credential-socket" pod "052e4f45-bd16-4ef9-9c17-10a331df3ad4" (UID: "052e4f45-bd16-4ef9-9c17-10a331df3ad4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.872021 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.871962    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istiod-ca-cert" pod "052e4f45-bd16-4ef9-9c17-10a331df3ad4" (UID: "052e4f45-bd16-4ef9-9c17-10a331df3ad4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:15.873834 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.873798    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-token" pod "052e4f45-bd16-4ef9-9c17-10a331df3ad4" (UID: "052e4f45-bd16-4ef9-9c17-10a331df3ad4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:15.874063 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.874035    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-envoy" pod "052e4f45-bd16-4ef9-9c17-10a331df3ad4" (UID: "052e4f45-bd16-4ef9-9c17-10a331df3ad4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:18:15.874465 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.874437    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-kube-api-access-w64rb" pod "052e4f45-bd16-4ef9-9c17-10a331df3ad4" (UID: "052e4f45-bd16-4ef9-9c17-10a331df3ad4"). InnerVolumeSpecName "kube-api-access-w64rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:15.875252 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.875225    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-podinfo" pod "052e4f45-bd16-4ef9-9c17-10a331df3ad4" (UID: "052e4f45-bd16-4ef9-9c17-10a331df3ad4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:18:15.972817 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.972744    4512 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-podinfo\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.972817 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.972781    4512 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-token\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.972817 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.972799    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w64rb\" (UniqueName: \"kubernetes.io/projected/052e4f45-bd16-4ef9-9c17-10a331df3ad4-kube-api-access-w64rb\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.972817 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.972813    4512 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istiod-ca-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.973051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.972827    4512 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-envoy\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.973051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.972840    4512 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-istio-data\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.973051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.972855    4512 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.973051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.972868    4512 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-workload-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:15.973051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:15.972881    4512 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052e4f45-bd16-4ef9-9c17-10a331df3ad4-credential-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:16.140861 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:16.140827    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:18:16.690779 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:16.690689    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"
Nov 04 02:18:16.690779 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:16.690715    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f" event={"ID":"052e4f45-bd16-4ef9-9c17-10a331df3ad4","Type":"ContainerDied","Data":"b2a56256f71b83faa36310c294e84d543b6a48dc6386e622bc2ab9cd04bb2d7d"}
Nov 04 02:18:16.691245 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:16.690780    4512 scope.go:122] "RemoveContainer" containerID="302c7287f172a1dfedfe4a9d40bfcf70e8781f04747895b08f01b0903a9971ed"
Nov 04 02:18:16.858276 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:16.858247    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"]
Nov 04 02:18:16.907962 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:16.907930    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f"]
Nov 04 02:18:17.237978 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.237938    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:17.491604 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.491482    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:17.590259 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.590209    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"]
Nov 04 02:18:17.592347 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.592311    4512 cpu_manager.go:414] "containerMap: removing container" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerName="istio-proxy"
Nov 04 02:18:17.592347 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.592346    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerName="istio-proxy"
Nov 04 02:18:17.592668 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.592435    4512 cpu_manager.go:414] "containerMap: removing container" podUID="3c26fc7d-912d-4d7e-8a21-b7b26942e361" containerName="revoke-keys"
Nov 04 02:18:17.592668 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.592444    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c26fc7d-912d-4d7e-8a21-b7b26942e361" containerName="revoke-keys"
Nov 04 02:18:17.592822 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.592800    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c26fc7d-912d-4d7e-8a21-b7b26942e361" containerName="revoke-keys"
Nov 04 02:18:17.592892 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.592845    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" containerName="istio-proxy"
Nov 04 02:18:17.615355 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.615321    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:18:17.645796 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.645766    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"]
Nov 04 02:18:17.672049 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.672022    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-3d082d-dockercfg-q7fzx\""
Nov 04 02:18:17.672222 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.672182    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-3d082d\""
Nov 04 02:18:17.703636 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.703607    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df582a65-4be0-4b85-ae0c-3111b00df6e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b\" (UID: \"df582a65-4be0-4b85-ae0c-3111b00df6e9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:18:17.703993 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.703821    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldmrq\" (UniqueName: \"kubernetes.io/projected/df582a65-4be0-4b85-ae0c-3111b00df6e9-kube-api-access-ldmrq\") pod \"payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b\" (UID: \"df582a65-4be0-4b85-ae0c-3111b00df6e9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:18:17.777819 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.777697    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"]
Nov 04 02:18:17.783948 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.783916    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:18:17.805285 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.805253    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df582a65-4be0-4b85-ae0c-3111b00df6e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b\" (UID: \"df582a65-4be0-4b85-ae0c-3111b00df6e9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:18:17.805399 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.805371    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmrq\" (UniqueName: \"kubernetes.io/projected/df582a65-4be0-4b85-ae0c-3111b00df6e9-kube-api-access-ldmrq\") pod \"payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b\" (UID: \"df582a65-4be0-4b85-ae0c-3111b00df6e9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:18:17.805929 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.805908    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df582a65-4be0-4b85-ae0c-3111b00df6e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b\" (UID: \"df582a65-4be0-4b85-ae0c-3111b00df6e9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:18:17.815462 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.815431    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"]
Nov 04 02:18:17.866762 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.866730    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldmrq\" (UniqueName: \"kubernetes.io/projected/df582a65-4be0-4b85-ae0c-3111b00df6e9-kube-api-access-ldmrq\") pod \"payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b\" (UID: \"df582a65-4be0-4b85-ae0c-3111b00df6e9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:18:17.906562 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.906529    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8fa3273-465a-4741-b8b9-3e98ad56ee46-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll\" (UID: \"a8fa3273-465a-4741-b8b9-3e98ad56ee46\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:18:17.906781 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.906567    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh9qp\" (UniqueName: \"kubernetes.io/projected/a8fa3273-465a-4741-b8b9-3e98ad56ee46-kube-api-access-qh9qp\") pod \"payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll\" (UID: \"a8fa3273-465a-4741-b8b9-3e98ad56ee46\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:18:17.932306 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:17.932275    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:18:18.010559 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.008463    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8fa3273-465a-4741-b8b9-3e98ad56ee46-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll\" (UID: \"a8fa3273-465a-4741-b8b9-3e98ad56ee46\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:18:18.010559 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.008563    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh9qp\" (UniqueName: \"kubernetes.io/projected/a8fa3273-465a-4741-b8b9-3e98ad56ee46-kube-api-access-qh9qp\") pod \"payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll\" (UID: \"a8fa3273-465a-4741-b8b9-3e98ad56ee46\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:18:18.010559 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.009499    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8fa3273-465a-4741-b8b9-3e98ad56ee46-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll\" (UID: \"a8fa3273-465a-4741-b8b9-3e98ad56ee46\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:18:18.070108 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.070036    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh9qp\" (UniqueName: \"kubernetes.io/projected/a8fa3273-465a-4741-b8b9-3e98ad56ee46-kube-api-access-qh9qp\") pod \"payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll\" (UID: \"a8fa3273-465a-4741-b8b9-3e98ad56ee46\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:18:18.100939 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.100908    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:18:18.110085 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:18.110062    4512 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-26d257c38a93: secret "agc-cred-26d257c38a93" not found
Nov 04 02:18:18.110213 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:18.110089    4512 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4: secret "agc-cred-26d257c38a93" not found
Nov 04 02:18:18.110213 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:18.110155    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials podName:1d3300f6-b801-481d-a7f9-b76114860e85 nodeName:}" failed. No retries permitted until 2026-11-04 02:18:50.110134927 +0000 UTC m=+1391.915591536 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials") pod "payload-processing-external-model-6945d56d5f-5trg4" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85") : secret "agc-cred-26d257c38a93" not found
Nov 04 02:18:18.316248 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.316206    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="052e4f45-bd16-4ef9-9c17-10a331df3ad4" path="/var/lib/kubelet/pods/052e4f45-bd16-4ef9-9c17-10a331df3ad4/volumes"
Nov 04 02:18:18.317564 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.317542    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"]
Nov 04 02:18:18.320736 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:18:18.320681    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf582a65_4be0_4b85_ae0c_3111b00df6e9.slice/crio-ec988994ac65791a92ace6255db303b1b057ec442460fc8ef7127f6ff9ecd3d2 WatchSource:0}: Error finding container ec988994ac65791a92ace6255db303b1b057ec442460fc8ef7127f6ff9ecd3d2: Status 404 returned error can't find the container with id ec988994ac65791a92ace6255db303b1b057ec442460fc8ef7127f6ff9ecd3d2
Nov 04 02:18:18.382624 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.379293    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:18.486271 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:18:18.486242    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8fa3273_465a_4741_b8b9_3e98ad56ee46.slice/crio-1267c858021c447211bbbd16780e49cf67cc4cd30f05db7ae41fb1e0f49405e5 WatchSource:0}: Error finding container 1267c858021c447211bbbd16780e49cf67cc4cd30f05db7ae41fb1e0f49405e5: Status 404 returned error can't find the container with id 1267c858021c447211bbbd16780e49cf67cc4cd30f05db7ae41fb1e0f49405e5
Nov 04 02:18:18.486414 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.486391    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"]
Nov 04 02:18:18.714297 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.714258    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b" event={"ID":"df582a65-4be0-4b85-ae0c-3111b00df6e9","Type":"ContainerStarted","Data":"c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68"}
Nov 04 02:18:18.714846 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.714322    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b" event={"ID":"df582a65-4be0-4b85-ae0c-3111b00df6e9","Type":"ContainerStarted","Data":"ec988994ac65791a92ace6255db303b1b057ec442460fc8ef7127f6ff9ecd3d2"}
Nov 04 02:18:18.714846 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.714370    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:18:18.716073 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.716050    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll" event={"ID":"a8fa3273-465a-4741-b8b9-3e98ad56ee46","Type":"ContainerStarted","Data":"801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16"}
Nov 04 02:18:18.716222 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.716084    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll" event={"ID":"a8fa3273-465a-4741-b8b9-3e98ad56ee46","Type":"ContainerStarted","Data":"1267c858021c447211bbbd16780e49cf67cc4cd30f05db7ae41fb1e0f49405e5"}
Nov 04 02:18:18.716222 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.716180    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:18:18.833966 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.833861    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b" podStartSLOduration=1.8338471699999999 podStartE2EDuration="1.83384717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:18:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:18:18.8292233 +0000 UTC m=+1360.634679932" watchObservedRunningTime="2026-11-04 02:18:18.83384717 +0000 UTC m=+1360.639303800"
Nov 04 02:18:18.949489 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:18.949432    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll" podStartSLOduration=1.94941678 podStartE2EDuration="1.94941678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:18:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:18:18.944614444 +0000 UTC m=+1360.750071091" watchObservedRunningTime="2026-11-04 02:18:18.94941678 +0000 UTC m=+1360.754873409"
Nov 04 02:18:19.367652 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:19.364307    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:19.567556 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:19.567520    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:20.397554 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:20.397516    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:21.360597 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.360554    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"]
Nov 04 02:18:21.365249 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.365225    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.417644 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.417607    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 04 02:18:21.417644 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.417611    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 04 02:18:21.418087 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.417643    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 04 02:18:21.418087 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.417714    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zf425\""
Nov 04 02:18:21.451036 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.450999    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgvh8\" (UniqueName: \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-kube-api-access-wgvh8\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.451193 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.451070    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.451193 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.451167    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.451193 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.451189    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.451395 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.451371    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.451449 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.451437    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1133414-79e3-4a43-b32e-680c0a115b30-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.451488 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.451458    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.472172 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.472136    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"]
Nov 04 02:18:21.552588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.552549    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1133414-79e3-4a43-b32e-680c0a115b30-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.552588 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.552588    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.552862 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.552682    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgvh8\" (UniqueName: \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-kube-api-access-wgvh8\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.552862 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.552729    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.552963 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.552882    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.552963 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.552914    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.553068 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.553043    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.553563 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.553499    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.555162 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.555134    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1133414-79e3-4a43-b32e-680c0a115b30-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.555319 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.555303    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.555414 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.555391    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.555519 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.555489    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.614166 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.614081    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgvh8\" (UniqueName: \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-kube-api-access-wgvh8\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.615003 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.614972    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6kjfp\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:21.679994 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:21.679958    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:22.122047 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:18:22.122013    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1133414_79e3_4a43_b32e_680c0a115b30.slice/crio-989f2d05cde92106c25598490a97a2e393eab80cad5f2d8e493b6b30bbb950f6 WatchSource:0}: Error finding container 989f2d05cde92106c25598490a97a2e393eab80cad5f2d8e493b6b30bbb950f6: Status 404 returned error can't find the container with id 989f2d05cde92106c25598490a97a2e393eab80cad5f2d8e493b6b30bbb950f6
Nov 04 02:18:22.123430 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:22.123399    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"]
Nov 04 02:18:22.749229 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:22.749196    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp" event={"ID":"f1133414-79e3-4a43-b32e-680c0a115b30","Type":"ContainerStarted","Data":"989f2d05cde92106c25598490a97a2e393eab80cad5f2d8e493b6b30bbb950f6"}
Nov 04 02:18:23.624856 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:23.624815    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"]
Nov 04 02:18:23.831042 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:23.831006    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:24.025540 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:24.025453    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:26.141918 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:26.141547    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:18:26.733058 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:26.733023    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:26.786100 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:26.786044    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:18:26.979165 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:26.979111    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:27.092567 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:27.092527    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:18:27.092683 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:27.092595    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:18:27.810693 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:27.810649    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp" event={"ID":"f1133414-79e3-4a43-b32e-680c0a115b30","Type":"ContainerStarted","Data":"e62d1549aed963a7b0a4cf8183b25f5c10745c04eaea5d2d1889feb812388c1b"}
Nov 04 02:18:27.811177 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:27.810730    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:28.057326 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.057260    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp" podStartSLOduration=2.090360231 podStartE2EDuration="7.057246695s" totalImagesPullingTime="4.966886464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:18:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:18:28.006931441 +0000 UTC m=+1369.812388072" watchObservedRunningTime="2026-11-04 02:18:28.057246695 +0000 UTC m=+1369.862703325"
Nov 04 02:18:28.625487 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.625436    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw" podUID="c0569531-49c1-412f-91d0-053f66157457" containerName="payload-processing" containerID="cri-o://8f6dce22dd517afdcb82664d007b6cfc963d5f228526d2b49c6de13dc5f48ba6" gracePeriod=25
Nov 04 02:18:28.829775 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.827756    4512 generic.go:356] "Generic (PLEG): container finished" podID="c0569531-49c1-412f-91d0-053f66157457" containerID="8f6dce22dd517afdcb82664d007b6cfc963d5f228526d2b49c6de13dc5f48ba6" exitCode=0
Nov 04 02:18:28.829775 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.828583    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw" event={"ID":"c0569531-49c1-412f-91d0-053f66157457","Type":"ContainerDied","Data":"8f6dce22dd517afdcb82664d007b6cfc963d5f228526d2b49c6de13dc5f48ba6"}
Nov 04 02:18:28.830791 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.830767    4512 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-6kjfp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:18:28.830931 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.830849    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp" podUID="f1133414-79e3-4a43-b32e-680c0a115b30" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:18:28.890353 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.890197    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:28.921762 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.921733    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:18:28.963207 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.963178    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-routing-overlay-volume\") pod \"c0569531-49c1-412f-91d0-053f66157457\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") "
Nov 04 02:18:28.963378 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.963260    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-plugins-config-volume\") pod \"c0569531-49c1-412f-91d0-053f66157457\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") "
Nov 04 02:18:28.963378 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.963290    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-provider-credentials\" (UniqueName: \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-provider-credentials\") pod \"c0569531-49c1-412f-91d0-053f66157457\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") "
Nov 04 02:18:28.963378 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.963324    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-kube-api-access-xhlgm\" (UniqueName: \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-kube-api-access-xhlgm\") pod \"c0569531-49c1-412f-91d0-053f66157457\" (UID: \"c0569531-49c1-412f-91d0-053f66157457\") "
Nov 04 02:18:28.963645 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.963618    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-routing-overlay-volume" pod "c0569531-49c1-412f-91d0-053f66157457" (UID: "c0569531-49c1-412f-91d0-053f66157457"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:28.963748 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.963683    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-plugins-config-volume" pod "c0569531-49c1-412f-91d0-053f66157457" (UID: "c0569531-49c1-412f-91d0-053f66157457"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:28.963891 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.963876    4512 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-routing-overlay-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:28.963925 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.963894    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0569531-49c1-412f-91d0-053f66157457-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:28.965777 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.965752    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-kube-api-access-xhlgm" pod "c0569531-49c1-412f-91d0-053f66157457" (UID: "c0569531-49c1-412f-91d0-053f66157457"). InnerVolumeSpecName "kube-api-access-xhlgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:28.965889 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:28.965778    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-provider-credentials" pod "c0569531-49c1-412f-91d0-053f66157457" (UID: "c0569531-49c1-412f-91d0-053f66157457"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:29.065649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:29.065610    4512 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-provider-credentials\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:29.065649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:29.065651    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhlgm\" (UniqueName: \"kubernetes.io/projected/c0569531-49c1-412f-91d0-053f66157457-kube-api-access-xhlgm\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:29.726421 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:29.726387    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:18:29.726624 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:29.726472    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:18:29.837765 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:29.837733    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"
Nov 04 02:18:29.838160 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:29.837767    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw" event={"ID":"c0569531-49c1-412f-91d0-053f66157457","Type":"ContainerDied","Data":"0f741fb105b7408f8947bbd37e23a7f02612554c959fd4d8e4ab3152fee1d8a9"}
Nov 04 02:18:29.838160 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:29.837825    4512 scope.go:122] "RemoveContainer" containerID="8f6dce22dd517afdcb82664d007b6cfc963d5f228526d2b49c6de13dc5f48ba6"
Nov 04 02:18:30.075002 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:30.074969    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"]
Nov 04 02:18:30.078698 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:30.078676    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-055a14/payload-processing-external-model-e2e-extmod-b-055a14-67fc29fnw"]
Nov 04 02:18:30.313240 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:30.313204    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0569531-49c1-412f-91d0-053f66157457" path="/var/lib/kubelet/pods/c0569531-49c1-412f-91d0-053f66157457/volumes"
Nov 04 02:18:31.829863 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:31.829830    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:18:31.932113 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:31.932073    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:32.969565 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:32.969526    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:33.217343 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:33.217307    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:33.519192 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:33.519151    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:33.779423 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:33.779313    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:36.140957 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:36.140918    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:18:38.568728 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:38.568694    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"]
Nov 04 02:18:38.675640 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:38.675606    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:38.675854 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:38.675830    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"]
Nov 04 02:18:38.992755 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:38.992704    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:39.242051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:39.242016    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:40.047229 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:40.047192    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:43.569750 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.569690    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6" podUID="902d38f5-d5c2-429c-9bb2-d39e34660dad" containerName="payload-processing" containerID="cri-o://2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791" gracePeriod=25
Nov 04 02:18:43.676168 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.676103    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll" podUID="adad2e2b-654d-459e-a642-308912801cb2" containerName="payload-pre-processing" containerID="cri-o://a3a3decff7c4976df1b178c14e20431df174d4054467d075481a74f60ddf0456" gracePeriod=25
Nov 04 02:18:43.837452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.837388    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:18:43.921158 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.921108    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:43.949180 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.949141    4512 generic.go:356] "Generic (PLEG): container finished" podID="902d38f5-d5c2-429c-9bb2-d39e34660dad" containerID="2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791" exitCode=0
Nov 04 02:18:43.949367 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.949190    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"
Nov 04 02:18:43.949367 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.949202    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6" event={"ID":"902d38f5-d5c2-429c-9bb2-d39e34660dad","Type":"ContainerDied","Data":"2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791"}
Nov 04 02:18:43.949367 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.949306    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6" event={"ID":"902d38f5-d5c2-429c-9bb2-d39e34660dad","Type":"ContainerDied","Data":"ed7ae5c87b63537a26eac180b9f35c5c24f891424e6ae0d1edcb10883d87df8a"}
Nov 04 02:18:43.949367 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.949340    4512 scope.go:122] "RemoveContainer" containerID="2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791"
Nov 04 02:18:43.950091 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.950068    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/902d38f5-d5c2-429c-9bb2-d39e34660dad-kube-api-access-54h6d\" (UniqueName: \"kubernetes.io/projected/902d38f5-d5c2-429c-9bb2-d39e34660dad-kube-api-access-54h6d\") pod \"902d38f5-d5c2-429c-9bb2-d39e34660dad\" (UID: \"902d38f5-d5c2-429c-9bb2-d39e34660dad\") "
Nov 04 02:18:43.950204 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.950180    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/902d38f5-d5c2-429c-9bb2-d39e34660dad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/902d38f5-d5c2-429c-9bb2-d39e34660dad-plugins-config-volume\") pod \"902d38f5-d5c2-429c-9bb2-d39e34660dad\" (UID: \"902d38f5-d5c2-429c-9bb2-d39e34660dad\") "
Nov 04 02:18:43.951663 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.951633    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/902d38f5-d5c2-429c-9bb2-d39e34660dad-plugins-config-volume" pod "902d38f5-d5c2-429c-9bb2-d39e34660dad" (UID: "902d38f5-d5c2-429c-9bb2-d39e34660dad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:43.952476 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.952441    4512 generic.go:356] "Generic (PLEG): container finished" podID="adad2e2b-654d-459e-a642-308912801cb2" containerID="a3a3decff7c4976df1b178c14e20431df174d4054467d075481a74f60ddf0456" exitCode=0
Nov 04 02:18:43.952594 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.952541    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll" event={"ID":"adad2e2b-654d-459e-a642-308912801cb2","Type":"ContainerDied","Data":"a3a3decff7c4976df1b178c14e20431df174d4054467d075481a74f60ddf0456"}
Nov 04 02:18:43.953355 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.953331    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/902d38f5-d5c2-429c-9bb2-d39e34660dad-kube-api-access-54h6d" pod "902d38f5-d5c2-429c-9bb2-d39e34660dad" (UID: "902d38f5-d5c2-429c-9bb2-d39e34660dad"). InnerVolumeSpecName "kube-api-access-54h6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:43.979175 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.979151    4512 scope.go:122] "RemoveContainer" containerID="2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791"
Nov 04 02:18:43.979529 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:43.979491    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791\": container with ID starting with 2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791 not found: ID does not exist" containerID="2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791"
Nov 04 02:18:43.979623 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.979542    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791"} err="failed to get container status \"2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791\": rpc error: code = NotFound desc = could not find container \"2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791\": container with ID starting with 2ea848c29b451b0cf1479733a847c7c42c6cf83f2351b0bbc8ad6719adc1e791 not found: ID does not exist"
Nov 04 02:18:43.979623 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:43.979613    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:18:44.051264 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.051234    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/adad2e2b-654d-459e-a642-308912801cb2-kube-api-access-88wv2\" (UniqueName: \"kubernetes.io/projected/adad2e2b-654d-459e-a642-308912801cb2-kube-api-access-88wv2\") pod \"adad2e2b-654d-459e-a642-308912801cb2\" (UID: \"adad2e2b-654d-459e-a642-308912801cb2\") "
Nov 04 02:18:44.051468 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.051338    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/adad2e2b-654d-459e-a642-308912801cb2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adad2e2b-654d-459e-a642-308912801cb2-plugins-config-volume\") pod \"adad2e2b-654d-459e-a642-308912801cb2\" (UID: \"adad2e2b-654d-459e-a642-308912801cb2\") "
Nov 04 02:18:44.051690 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.051669    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/902d38f5-d5c2-429c-9bb2-d39e34660dad-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:44.051690 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.051689    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54h6d\" (UniqueName: \"kubernetes.io/projected/902d38f5-d5c2-429c-9bb2-d39e34660dad-kube-api-access-54h6d\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:44.051832 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.051714    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adad2e2b-654d-459e-a642-308912801cb2-plugins-config-volume" pod "adad2e2b-654d-459e-a642-308912801cb2" (UID: "adad2e2b-654d-459e-a642-308912801cb2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:18:44.053516 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.053487    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adad2e2b-654d-459e-a642-308912801cb2-kube-api-access-88wv2" pod "adad2e2b-654d-459e-a642-308912801cb2" (UID: "adad2e2b-654d-459e-a642-308912801cb2"). InnerVolumeSpecName "kube-api-access-88wv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:18:44.152864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.152785    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88wv2\" (UniqueName: \"kubernetes.io/projected/adad2e2b-654d-459e-a642-308912801cb2-kube-api-access-88wv2\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:44.152864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.152825    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adad2e2b-654d-459e-a642-308912801cb2-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:18:44.372921 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.372883    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"]
Nov 04 02:18:44.378445 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.378416    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6"]
Nov 04 02:18:44.961639 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.961565    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"
Nov 04 02:18:44.961639 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.961579    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll" event={"ID":"adad2e2b-654d-459e-a642-308912801cb2","Type":"ContainerDied","Data":"a9101c1fbdedb9e996bf9030c6ce691ad79f83f5807c1cb4421ab60ef78476b9"}
Nov 04 02:18:44.961639 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:44.961638    4512 scope.go:122] "RemoveContainer" containerID="a3a3decff7c4976df1b178c14e20431df174d4054467d075481a74f60ddf0456"
Nov 04 02:18:45.127365 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:45.127334    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"]
Nov 04 02:18:45.177017 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:45.176987    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll"]
Nov 04 02:18:45.952283 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:45.952242    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:46.141243 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:46.141197    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:18:46.314481 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:46.314448    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="902d38f5-d5c2-429c-9bb2-d39e34660dad" path="/var/lib/kubelet/pods/902d38f5-d5c2-429c-9bb2-d39e34660dad/volumes"
Nov 04 02:18:46.314902 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:46.314886    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="adad2e2b-654d-459e-a642-308912801cb2" path="/var/lib/kubelet/pods/adad2e2b-654d-459e-a642-308912801cb2/volumes"
Nov 04 02:18:48.304073 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:48.304040    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:50.082807 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:50.082699    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:50.134038 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:50.134005    4512 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-26d257c38a93: secret "agc-cred-26d257c38a93" not found
Nov 04 02:18:50.134038 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:50.134039    4512 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4: secret "agc-cred-26d257c38a93" not found
Nov 04 02:18:50.134257 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:50.134091    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials podName:1d3300f6-b801-481d-a7f9-b76114860e85 nodeName:}" failed. No retries permitted until 2026-11-04 02:19:54.134078337 +0000 UTC m=+1455.939534946 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials") pod "payload-processing-external-model-6945d56d5f-5trg4" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85") : secret "agc-cred-26d257c38a93" not found
Nov 04 02:18:51.757970 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:51.757934    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:51.964478 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:51.964430    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:53.355456 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:53.355408    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:53.870564 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:53.870498    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:54.698394 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:54.698357    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:55.305713 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:55.305676    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:18:55.369684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:18:55.369647    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:56.779679 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:18:56.779638    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:19:00.644799 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:00.644753    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:00.849064 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:00.849023    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:01.048452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:01.048404    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:01.952577 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:01.952537    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"]
Nov 04 02:19:02.062790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:02.062756    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"]
Nov 04 02:19:02.378861 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:02.378808    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:02.582826 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:02.582788    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:05.306197 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:05.306164    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:19:05.369124 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:05.369072    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:06.927333 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:06.927288    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:06.953468 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:06.953424    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p" podUID="56b6b1b0-b1b3-48db-aa50-3ee10efcff1b" containerName="payload-processing" containerID="cri-o://1100b7b092d7012ed19ec47bd784a95969f4220b0635bb7266cc93b59b8f8838" gracePeriod=25
Nov 04 02:19:07.064064 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.064012    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw" podUID="b549b289-3be5-495c-8caf-df4e1e9934bf" containerName="payload-pre-processing" containerID="cri-o://a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b" gracePeriod=25
Nov 04 02:19:07.133245 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.133212    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:07.151679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.151500    4512 generic.go:356] "Generic (PLEG): container finished" podID="56b6b1b0-b1b3-48db-aa50-3ee10efcff1b" containerID="1100b7b092d7012ed19ec47bd784a95969f4220b0635bb7266cc93b59b8f8838" exitCode=0
Nov 04 02:19:07.151679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.151594    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p" event={"ID":"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b","Type":"ContainerDied","Data":"1100b7b092d7012ed19ec47bd784a95969f4220b0635bb7266cc93b59b8f8838"}
Nov 04 02:19:07.244878 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.244852    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:19:07.335546 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.335481    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:07.357750 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.357725    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:19:07.388616 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.388582    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-kube-api-access-kvbcv\" (UniqueName: \"kubernetes.io/projected/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-kube-api-access-kvbcv\") pod \"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b\" (UID: \"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b\") "
Nov 04 02:19:07.388823 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.388807    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-plugins-config-volume\") pod \"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b\" (UID: \"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b\") "
Nov 04 02:19:07.389230 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.389190    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-plugins-config-volume" pod "56b6b1b0-b1b3-48db-aa50-3ee10efcff1b" (UID: "56b6b1b0-b1b3-48db-aa50-3ee10efcff1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:19:07.389331 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.389318    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:07.390900 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.390870    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-kube-api-access-kvbcv" pod "56b6b1b0-b1b3-48db-aa50-3ee10efcff1b" (UID: "56b6b1b0-b1b3-48db-aa50-3ee10efcff1b"). InnerVolumeSpecName "kube-api-access-kvbcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:19:07.490084 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.490046    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b549b289-3be5-495c-8caf-df4e1e9934bf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b549b289-3be5-495c-8caf-df4e1e9934bf-plugins-config-volume\") pod \"b549b289-3be5-495c-8caf-df4e1e9934bf\" (UID: \"b549b289-3be5-495c-8caf-df4e1e9934bf\") "
Nov 04 02:19:07.490297 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.490171    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b549b289-3be5-495c-8caf-df4e1e9934bf-kube-api-access-rng59\" (UniqueName: \"kubernetes.io/projected/b549b289-3be5-495c-8caf-df4e1e9934bf-kube-api-access-rng59\") pod \"b549b289-3be5-495c-8caf-df4e1e9934bf\" (UID: \"b549b289-3be5-495c-8caf-df4e1e9934bf\") "
Nov 04 02:19:07.490543 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.490495    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b549b289-3be5-495c-8caf-df4e1e9934bf-plugins-config-volume" pod "b549b289-3be5-495c-8caf-df4e1e9934bf" (UID: "b549b289-3be5-495c-8caf-df4e1e9934bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:19:07.491048 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.491016    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvbcv\" (UniqueName: \"kubernetes.io/projected/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b-kube-api-access-kvbcv\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:07.491048 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.491047    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b549b289-3be5-495c-8caf-df4e1e9934bf-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:07.493480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.493447    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b549b289-3be5-495c-8caf-df4e1e9934bf-kube-api-access-rng59" pod "b549b289-3be5-495c-8caf-df4e1e9934bf" (UID: "b549b289-3be5-495c-8caf-df4e1e9934bf"). InnerVolumeSpecName "kube-api-access-rng59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:19:07.592428 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:07.592389    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rng59\" (UniqueName: \"kubernetes.io/projected/b549b289-3be5-495c-8caf-df4e1e9934bf-kube-api-access-rng59\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:08.160381 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.160343    4512 generic.go:356] "Generic (PLEG): container finished" podID="b549b289-3be5-495c-8caf-df4e1e9934bf" containerID="a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b" exitCode=0
Nov 04 02:19:08.160907 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.160392    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"
Nov 04 02:19:08.160907 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.160442    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw" event={"ID":"b549b289-3be5-495c-8caf-df4e1e9934bf","Type":"ContainerDied","Data":"a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b"}
Nov 04 02:19:08.160907 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.160501    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw" event={"ID":"b549b289-3be5-495c-8caf-df4e1e9934bf","Type":"ContainerDied","Data":"8b52a5e505741d7955a9aa47f745044134b43fd156a272a28b184faa649f32e0"}
Nov 04 02:19:08.160907 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.160553    4512 scope.go:122] "RemoveContainer" containerID="a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b"
Nov 04 02:19:08.162233 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.162210    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"
Nov 04 02:19:08.162233 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.162221    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p" event={"ID":"56b6b1b0-b1b3-48db-aa50-3ee10efcff1b","Type":"ContainerDied","Data":"cc1fa63f0cf16a3a9a1519f95369dbeaa0688152562819579bc2d76ce335be9a"}
Nov 04 02:19:08.173195 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.173173    4512 scope.go:122] "RemoveContainer" containerID="a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b"
Nov 04 02:19:08.173582 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:19:08.173553    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b\": container with ID starting with a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b not found: ID does not exist" containerID="a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b"
Nov 04 02:19:08.173727 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.173592    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b"} err="failed to get container status \"a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b\": rpc error: code = NotFound desc = could not find container \"a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b\": container with ID starting with a07ebad865287aef9559caa261dcbc458ccfbd5176cf55c22093c13fd0198b0b not found: ID does not exist"
Nov 04 02:19:08.173727 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.173611    4512 scope.go:122] "RemoveContainer" containerID="1100b7b092d7012ed19ec47bd784a95969f4220b0635bb7266cc93b59b8f8838"
Nov 04 02:19:08.328879 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.328841    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"]
Nov 04 02:19:08.379866 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.379832    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw"]
Nov 04 02:19:08.444358 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.444297    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"]
Nov 04 02:19:08.450366 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.450332    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p"]
Nov 04 02:19:08.486556 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.486526    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:08.978081 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:08.978049    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:09.454501 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:09.454457    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:09.658268 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:09.658224    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:10.080210 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080177    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"]
Nov 04 02:19:10.080880 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080867    4512 cpu_manager.go:414] "containerMap: removing container" podUID="adad2e2b-654d-459e-a642-308912801cb2" containerName="payload-pre-processing"
Nov 04 02:19:10.080934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080882    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="adad2e2b-654d-459e-a642-308912801cb2" containerName="payload-pre-processing"
Nov 04 02:19:10.080934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080908    4512 cpu_manager.go:414] "containerMap: removing container" podUID="56b6b1b0-b1b3-48db-aa50-3ee10efcff1b" containerName="payload-processing"
Nov 04 02:19:10.080934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080913    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56b6b1b0-b1b3-48db-aa50-3ee10efcff1b" containerName="payload-processing"
Nov 04 02:19:10.080934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080924    4512 cpu_manager.go:414] "containerMap: removing container" podUID="b549b289-3be5-495c-8caf-df4e1e9934bf" containerName="payload-pre-processing"
Nov 04 02:19:10.080934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080929    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b549b289-3be5-495c-8caf-df4e1e9934bf" containerName="payload-pre-processing"
Nov 04 02:19:10.081089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080945    4512 cpu_manager.go:414] "containerMap: removing container" podUID="902d38f5-d5c2-429c-9bb2-d39e34660dad" containerName="payload-processing"
Nov 04 02:19:10.081089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080950    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="902d38f5-d5c2-429c-9bb2-d39e34660dad" containerName="payload-processing"
Nov 04 02:19:10.081089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080974    4512 cpu_manager.go:414] "containerMap: removing container" podUID="c0569531-49c1-412f-91d0-053f66157457" containerName="payload-processing"
Nov 04 02:19:10.081089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.080979    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0569531-49c1-412f-91d0-053f66157457" containerName="payload-processing"
Nov 04 02:19:10.081089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.081058    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="b549b289-3be5-495c-8caf-df4e1e9934bf" containerName="payload-pre-processing"
Nov 04 02:19:10.081089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.081067    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="902d38f5-d5c2-429c-9bb2-d39e34660dad" containerName="payload-processing"
Nov 04 02:19:10.081089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.081078    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0569531-49c1-412f-91d0-053f66157457" containerName="payload-processing"
Nov 04 02:19:10.081089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.081084    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="adad2e2b-654d-459e-a642-308912801cb2" containerName="payload-pre-processing"
Nov 04 02:19:10.081309 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.081092    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="56b6b1b0-b1b3-48db-aa50-3ee10efcff1b" containerName="payload-processing"
Nov 04 02:19:10.085934 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.085914    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.137706 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.137673    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c590b63a-k7qmd\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.137706 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.137718    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c590b63a-k7qmd\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.137943 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.137894    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mrpt\" (UniqueName: \"kubernetes.io/projected/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-kube-api-access-8mrpt\") pod \"maas-api-revoke-sub-c590b63a-k7qmd\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.142111 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.142086    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"]
Nov 04 02:19:10.171074 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.171034    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:19:10.186054 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.186023    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:19:10.238793 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.238756    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c590b63a-k7qmd\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.238793 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.238798    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c590b63a-k7qmd\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.239020 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.238897    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mrpt\" (UniqueName: \"kubernetes.io/projected/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-kube-api-access-8mrpt\") pod \"maas-api-revoke-sub-c590b63a-k7qmd\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.239566 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.239501    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c590b63a-k7qmd\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.241304 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.241285    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c590b63a-k7qmd\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.300400 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.300360    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mrpt\" (UniqueName: \"kubernetes.io/projected/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-kube-api-access-8mrpt\") pod \"maas-api-revoke-sub-c590b63a-k7qmd\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.326540 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.326472    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56b6b1b0-b1b3-48db-aa50-3ee10efcff1b" path="/var/lib/kubelet/pods/56b6b1b0-b1b3-48db-aa50-3ee10efcff1b/volumes"
Nov 04 02:19:10.327343 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.327314    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b549b289-3be5-495c-8caf-df4e1e9934bf" path="/var/lib/kubelet/pods/b549b289-3be5-495c-8caf-df4e1e9934bf/volumes"
Nov 04 02:19:10.401078 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.400987    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:10.685941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.685849    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:10.786280 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.786252    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"]
Nov 04 02:19:10.885339 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:10.885301    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:11.165857 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:11.165821    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:11.191459 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:11.191419    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd" event={"ID":"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3","Type":"ContainerStarted","Data":"daa8e0f0e57114a5661e81e66a0afc58495e4a0df1ccaed7b7ddfc4eac567fce"}
Nov 04 02:19:11.191654 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:11.191485    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd" event={"ID":"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3","Type":"ContainerStarted","Data":"106a2a4119f6f0615560c3ebeb6ba07b42363058b308265b39d1cc9d1291c8f6"}
Nov 04 02:19:11.308894 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:11.308758    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd" podStartSLOduration=1.308734073 podStartE2EDuration="1.308734073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:19:11.306419007 +0000 UTC m=+1413.111875647" watchObservedRunningTime="2026-11-04 02:19:11.308734073 +0000 UTC m=+1413.114190714"
Nov 04 02:19:11.394013 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:11.393970    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:11.592651 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:11.592556    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:12.202436 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:12.202402    4512 generic.go:356] "Generic (PLEG): container finished" podID="ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3" containerID="daa8e0f0e57114a5661e81e66a0afc58495e4a0df1ccaed7b7ddfc4eac567fce" exitCode=0
Nov 04 02:19:12.202915 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:12.202498    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd" event={"ID":"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3","Type":"ContainerDied","Data":"daa8e0f0e57114a5661e81e66a0afc58495e4a0df1ccaed7b7ddfc4eac567fce"}
Nov 04 02:19:12.312281 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:12.312241    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:12.538182 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:12.538143    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:12.740044 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:12.740005    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:12.941184 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:12.941082    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:13.367084 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:13.367057    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:13.482013 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:13.481972    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-kube-api-access-8mrpt\" (UniqueName: \"kubernetes.io/projected/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-kube-api-access-8mrpt\") pod \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") "
Nov 04 02:19:13.482214 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:13.482082    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-xks-ca\") pod \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") "
Nov 04 02:19:13.482294 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:13.482261    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-openshift-ca\") pod \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\" (UID: \"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3\") "
Nov 04 02:19:13.483056 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:13.482681    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-openshift-ca" pod "ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3" (UID: "ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:19:13.483438 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:13.483394    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:13.485392 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:13.485353    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-kube-api-access-8mrpt" pod "ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3" (UID: "ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3"). InnerVolumeSpecName "kube-api-access-8mrpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:19:13.485688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:13.485665    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-xks-ca" pod "ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3" (UID: "ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:19:13.584875 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:13.584845    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mrpt\" (UniqueName: \"kubernetes.io/projected/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-kube-api-access-8mrpt\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:13.584875 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:13.584872    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:14.218945 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:14.218904    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"
Nov 04 02:19:14.218945 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:14.218932    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd" event={"ID":"ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3","Type":"ContainerDied","Data":"106a2a4119f6f0615560c3ebeb6ba07b42363058b308265b39d1cc9d1291c8f6"}
Nov 04 02:19:14.219190 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:14.219029    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="106a2a4119f6f0615560c3ebeb6ba07b42363058b308265b39d1cc9d1291c8f6"
Nov 04 02:19:14.607354 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:14.607311    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:15.306046 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:15.306008    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:19:15.820873 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:15.820843    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:18.832395 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:18.832355    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:19.082847 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:19.082759    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:21.653716 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.653676    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"]
Nov 04 02:19:21.654650 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.654619    4512 cpu_manager.go:414] "containerMap: removing container" podUID="ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3" containerName="revoke-keys"
Nov 04 02:19:21.654650 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.654647    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3" containerName="revoke-keys"
Nov 04 02:19:21.654854 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.654779    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3" containerName="revoke-keys"
Nov 04 02:19:21.658837 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.658809    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:21.715979 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.715939    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"]
Nov 04 02:19:21.759944 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.759908    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:19:21.760126 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.760114    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:19:21.804719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.804686    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh62q\" (UniqueName: \"kubernetes.io/projected/885e2d43-195b-4be3-8459-99634df20617-kube-api-access-mh62q\") pod \"maas-api-revoke-sub-fb9f3d5a-9gmh9\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:21.804908 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.804845    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/885e2d43-195b-4be3-8459-99634df20617-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fb9f3d5a-9gmh9\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:21.804973 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.804921    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/885e2d43-195b-4be3-8459-99634df20617-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fb9f3d5a-9gmh9\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:21.906187 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.906092    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/885e2d43-195b-4be3-8459-99634df20617-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fb9f3d5a-9gmh9\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:21.906368 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.906278    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/885e2d43-195b-4be3-8459-99634df20617-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fb9f3d5a-9gmh9\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:21.906450 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.906381    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh62q\" (UniqueName: \"kubernetes.io/projected/885e2d43-195b-4be3-8459-99634df20617-kube-api-access-mh62q\") pod \"maas-api-revoke-sub-fb9f3d5a-9gmh9\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:21.907029 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.907003    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/885e2d43-195b-4be3-8459-99634df20617-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fb9f3d5a-9gmh9\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:21.908982 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.908946    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/885e2d43-195b-4be3-8459-99634df20617-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fb9f3d5a-9gmh9\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:21.975795 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:21.975765    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh62q\" (UniqueName: \"kubernetes.io/projected/885e2d43-195b-4be3-8459-99634df20617-kube-api-access-mh62q\") pod \"maas-api-revoke-sub-fb9f3d5a-9gmh9\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:22.274473 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:22.274432    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:22.579658 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:22.579564    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:22.661061 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:22.661030    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"]
Nov 04 02:19:22.661907 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:19:22.661882    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885e2d43_195b_4be3_8459_99634df20617.slice/crio-ce8c9326b462a2dfc9c37b378114ed1a8f6abfe6fe9ac4e141c0b28b9204a6b2 WatchSource:0}: Error finding container ce8c9326b462a2dfc9c37b378114ed1a8f6abfe6fe9ac4e141c0b28b9204a6b2: Status 404 returned error can't find the container with id ce8c9326b462a2dfc9c37b378114ed1a8f6abfe6fe9ac4e141c0b28b9204a6b2
Nov 04 02:19:22.783638 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:22.783602    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:23.294468 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:23.294431    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9" event={"ID":"885e2d43-195b-4be3-8459-99634df20617","Type":"ContainerStarted","Data":"6809bea27587ab8a9ce1455bfb32e5875bb8caa60e5a85f3af5e8b4d46a56b69"}
Nov 04 02:19:23.294685 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:23.294488    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9" event={"ID":"885e2d43-195b-4be3-8459-99634df20617","Type":"ContainerStarted","Data":"ce8c9326b462a2dfc9c37b378114ed1a8f6abfe6fe9ac4e141c0b28b9204a6b2"}
Nov 04 02:19:23.412957 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:23.412897    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9" podStartSLOduration=2.412883399 podStartE2EDuration="2.412883399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:19:23.409386596 +0000 UTC m=+1425.214843229" watchObservedRunningTime="2026-11-04 02:19:23.412883399 +0000 UTC m=+1425.218340031"
Nov 04 02:19:23.492319 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:23.492279    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"]
Nov 04 02:19:23.603792 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:23.603754    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"]
Nov 04 02:19:23.931240 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:23.929799    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:24.130294 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:24.130248    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:24.305899 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:24.305862    4512 generic.go:356] "Generic (PLEG): container finished" podID="885e2d43-195b-4be3-8459-99634df20617" containerID="6809bea27587ab8a9ce1455bfb32e5875bb8caa60e5a85f3af5e8b4d46a56b69" exitCode=0
Nov 04 02:19:24.313952 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:24.313918    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9" event={"ID":"885e2d43-195b-4be3-8459-99634df20617","Type":"ContainerDied","Data":"6809bea27587ab8a9ce1455bfb32e5875bb8caa60e5a85f3af5e8b4d46a56b69"}
Nov 04 02:19:25.309337 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.309293    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:19:25.504580 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.504548    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:25.662941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.662837    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/885e2d43-195b-4be3-8459-99634df20617-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/885e2d43-195b-4be3-8459-99634df20617-maas-api-xks-ca\") pod \"885e2d43-195b-4be3-8459-99634df20617\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") "
Nov 04 02:19:25.663122 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.662960    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/885e2d43-195b-4be3-8459-99634df20617-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/885e2d43-195b-4be3-8459-99634df20617-maas-api-openshift-ca\") pod \"885e2d43-195b-4be3-8459-99634df20617\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") "
Nov 04 02:19:25.663122 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.663003    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/885e2d43-195b-4be3-8459-99634df20617-kube-api-access-mh62q\" (UniqueName: \"kubernetes.io/projected/885e2d43-195b-4be3-8459-99634df20617-kube-api-access-mh62q\") pod \"885e2d43-195b-4be3-8459-99634df20617\" (UID: \"885e2d43-195b-4be3-8459-99634df20617\") "
Nov 04 02:19:25.663403 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.663364    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/885e2d43-195b-4be3-8459-99634df20617-maas-api-openshift-ca" pod "885e2d43-195b-4be3-8459-99634df20617" (UID: "885e2d43-195b-4be3-8459-99634df20617"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:19:25.663633 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.663614    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/885e2d43-195b-4be3-8459-99634df20617-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:25.665797 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.665772    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/885e2d43-195b-4be3-8459-99634df20617-maas-api-xks-ca" pod "885e2d43-195b-4be3-8459-99634df20617" (UID: "885e2d43-195b-4be3-8459-99634df20617"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:19:25.665797 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.665782    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/885e2d43-195b-4be3-8459-99634df20617-kube-api-access-mh62q" pod "885e2d43-195b-4be3-8459-99634df20617" (UID: "885e2d43-195b-4be3-8459-99634df20617"). InnerVolumeSpecName "kube-api-access-mh62q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:19:25.765228 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.765180    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/885e2d43-195b-4be3-8459-99634df20617-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:25.765228 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:25.765220    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mh62q\" (UniqueName: \"kubernetes.io/projected/885e2d43-195b-4be3-8459-99634df20617-kube-api-access-mh62q\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:26.334838 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:26.334805    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"
Nov 04 02:19:26.335249 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:26.334845    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9" event={"ID":"885e2d43-195b-4be3-8459-99634df20617","Type":"ContainerDied","Data":"ce8c9326b462a2dfc9c37b378114ed1a8f6abfe6fe9ac4e141c0b28b9204a6b2"}
Nov 04 02:19:26.335249 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:26.334900    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce8c9326b462a2dfc9c37b378114ed1a8f6abfe6fe9ac4e141c0b28b9204a6b2"
Nov 04 02:19:26.779395 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:19:26.779355    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:19:27.970368 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:27.970325    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:28.492886 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:28.492835    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h" podUID="891a1e83-0481-4fdf-bc83-91c61b411557" containerName="payload-processing" containerID="cri-o://9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec" gracePeriod=25
Nov 04 02:19:28.504619 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:28.504540    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:28.604320 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:28.604226    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j" podUID="f6227bc3-6dcb-46e4-b990-07c9ad43252a" containerName="payload-pre-processing" containerID="cri-o://d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892" gracePeriod=25
Nov 04 02:19:28.776773 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:28.776745    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:19:28.905784 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:28.905751    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/891a1e83-0481-4fdf-bc83-91c61b411557-kube-api-access-xvp4c\" (UniqueName: \"kubernetes.io/projected/891a1e83-0481-4fdf-bc83-91c61b411557-kube-api-access-xvp4c\") pod \"891a1e83-0481-4fdf-bc83-91c61b411557\" (UID: \"891a1e83-0481-4fdf-bc83-91c61b411557\") "
Nov 04 02:19:28.905977 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:28.905894    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/891a1e83-0481-4fdf-bc83-91c61b411557-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/891a1e83-0481-4fdf-bc83-91c61b411557-plugins-config-volume\") pod \"891a1e83-0481-4fdf-bc83-91c61b411557\" (UID: \"891a1e83-0481-4fdf-bc83-91c61b411557\") "
Nov 04 02:19:28.906399 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:28.906362    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/891a1e83-0481-4fdf-bc83-91c61b411557-plugins-config-volume" pod "891a1e83-0481-4fdf-bc83-91c61b411557" (UID: "891a1e83-0481-4fdf-bc83-91c61b411557"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:19:28.913345 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:28.913309    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/891a1e83-0481-4fdf-bc83-91c61b411557-kube-api-access-xvp4c" pod "891a1e83-0481-4fdf-bc83-91c61b411557" (UID: "891a1e83-0481-4fdf-bc83-91c61b411557"). InnerVolumeSpecName "kube-api-access-xvp4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:19:29.008091 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.007990    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvp4c\" (UniqueName: \"kubernetes.io/projected/891a1e83-0481-4fdf-bc83-91c61b411557-kube-api-access-xvp4c\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:29.008091 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.008028    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/891a1e83-0481-4fdf-bc83-91c61b411557-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:29.358471 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.358366    4512 generic.go:356] "Generic (PLEG): container finished" podID="891a1e83-0481-4fdf-bc83-91c61b411557" containerID="9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec" exitCode=0
Nov 04 02:19:29.358471 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.358420    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"
Nov 04 02:19:29.358471 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.358459    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h" event={"ID":"891a1e83-0481-4fdf-bc83-91c61b411557","Type":"ContainerDied","Data":"9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec"}
Nov 04 02:19:29.358766 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.358556    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h" event={"ID":"891a1e83-0481-4fdf-bc83-91c61b411557","Type":"ContainerDied","Data":"9574c14a92643427f1703b7b8b1e9a3415e538b7842044c4467c79086bb25b1c"}
Nov 04 02:19:29.358766 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.358585    4512 scope.go:122] "RemoveContainer" containerID="9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec"
Nov 04 02:19:29.372589 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.372562    4512 scope.go:122] "RemoveContainer" containerID="9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec"
Nov 04 02:19:29.373495 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:19:29.373469    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec\": container with ID starting with 9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec not found: ID does not exist" containerID="9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec"
Nov 04 02:19:29.373580 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.373540    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec"} err="failed to get container status \"9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec\": rpc error: code = NotFound desc = could not find container \"9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec\": container with ID starting with 9d68b79de08499eb46bd10eb8fa759f2cf964ff3fb45b902639a992a795345ec not found: ID does not exist"
Nov 04 02:19:29.526161 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.526104    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"]
Nov 04 02:19:29.575984 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:29.575950    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h"]
Nov 04 02:19:30.315604 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:30.315570    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="891a1e83-0481-4fdf-bc83-91c61b411557" path="/var/lib/kubelet/pods/891a1e83-0481-4fdf-bc83-91c61b411557/volumes"
Nov 04 02:19:31.114381 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:31.114338    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j" podUID="f6227bc3-6dcb-46e4-b990-07c9ad43252a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:19:32.280314 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:32.280283    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:32.537293 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:32.537252    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:32.862447 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:32.862363    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:34.386670 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:34.386632    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:35.305816 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:35.305774    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:19:37.682212 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:37.682162    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:41.114051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:41.114001    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j" podUID="f6227bc3-6dcb-46e4-b990-07c9ad43252a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:19:41.976082 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:41.976041    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:43.796338 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:43.796281    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:45.306042 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:45.306000    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:19:48.798939 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:48.798783    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:48.879940 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:48.879869    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:19:48.988292 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:48.988257    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6227bc3-6dcb-46e4-b990-07c9ad43252a-kube-api-access-t6pvf\" (UniqueName: \"kubernetes.io/projected/f6227bc3-6dcb-46e4-b990-07c9ad43252a-kube-api-access-t6pvf\") pod \"f6227bc3-6dcb-46e4-b990-07c9ad43252a\" (UID: \"f6227bc3-6dcb-46e4-b990-07c9ad43252a\") "
Nov 04 02:19:48.988452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:48.988391    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6227bc3-6dcb-46e4-b990-07c9ad43252a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6227bc3-6dcb-46e4-b990-07c9ad43252a-plugins-config-volume\") pod \"f6227bc3-6dcb-46e4-b990-07c9ad43252a\" (UID: \"f6227bc3-6dcb-46e4-b990-07c9ad43252a\") "
Nov 04 02:19:48.988876 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:48.988851    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6227bc3-6dcb-46e4-b990-07c9ad43252a-plugins-config-volume" pod "f6227bc3-6dcb-46e4-b990-07c9ad43252a" (UID: "f6227bc3-6dcb-46e4-b990-07c9ad43252a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:19:48.989133 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:48.988957    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6227bc3-6dcb-46e4-b990-07c9ad43252a-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:48.990586 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:48.990561    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6227bc3-6dcb-46e4-b990-07c9ad43252a-kube-api-access-t6pvf" pod "f6227bc3-6dcb-46e4-b990-07c9ad43252a" (UID: "f6227bc3-6dcb-46e4-b990-07c9ad43252a"). InnerVolumeSpecName "kube-api-access-t6pvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:19:49.090269 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.090222    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6pvf\" (UniqueName: \"kubernetes.io/projected/f6227bc3-6dcb-46e4-b990-07c9ad43252a-kube-api-access-t6pvf\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:19:49.092716 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.092683    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:49.292977 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.292934    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:49.594709 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.594615    4512 generic.go:356] "Generic (PLEG): container finished" podID="f6227bc3-6dcb-46e4-b990-07c9ad43252a" containerID="d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892" exitCode=0
Nov 04 02:19:49.594709 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.594684    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"
Nov 04 02:19:49.594935 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.594698    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j" event={"ID":"f6227bc3-6dcb-46e4-b990-07c9ad43252a","Type":"ContainerDied","Data":"d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892"}
Nov 04 02:19:49.594935 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.594767    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j" event={"ID":"f6227bc3-6dcb-46e4-b990-07c9ad43252a","Type":"ContainerDied","Data":"dc32f3a17f1c799616aea01476e20bc6aa8ead95dcc5609e3366d6c795b51b31"}
Nov 04 02:19:49.594935 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.594792    4512 scope.go:122] "RemoveContainer" containerID="d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892"
Nov 04 02:19:49.607821 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.607804    4512 scope.go:122] "RemoveContainer" containerID="d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892"
Nov 04 02:19:49.608115 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:19:49.608094    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892\": container with ID starting with d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892 not found: ID does not exist" containerID="d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892"
Nov 04 02:19:49.608169 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.608122    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892"} err="failed to get container status \"d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892\": rpc error: code = NotFound desc = could not find container \"d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892\": container with ID starting with d4a966945c09d03ac51816c1d0cc451e7e7a36ab13c286ac8155ae8720a84892 not found: ID does not exist"
Nov 04 02:19:49.760915 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.760880    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"]
Nov 04 02:19:49.810455 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:49.810422    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j"]
Nov 04 02:19:50.314001 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:50.313970    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6227bc3-6dcb-46e4-b990-07c9ad43252a" path="/var/lib/kubelet/pods/f6227bc3-6dcb-46e4-b990-07c9ad43252a/volumes"
Nov 04 02:19:50.921421 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:50.921386    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:51.122206 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:51.122171    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:51.987629 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:51.987593    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:54.016703 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:54.016664    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:54.153302 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:19:54.153264    4512 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-26d257c38a93: secret "agc-cred-26d257c38a93" not found
Nov 04 02:19:54.153302 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:19:54.153298    4512 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4: secret "agc-cred-26d257c38a93" not found
Nov 04 02:19:54.153604 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:19:54.153379    4512 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials podName:1d3300f6-b801-481d-a7f9-b76114860e85 nodeName:}" failed. No retries permitted until 2026-11-04 02:21:56.153359429 +0000 UTC m=+1577.958816053 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials") pod "payload-processing-external-model-6945d56d5f-5trg4" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85") : secret "agc-cred-26d257c38a93" not found
Nov 04 02:19:55.065216 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:55.065175    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:55.306253 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:55.306186    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:19:55.562689 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:55.562646    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:56.778732 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:19:56.778697    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:19:58.267900 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:58.267865    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:58.515312 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:58.515263    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:59.345488 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:19:59.345448    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:02.154846 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:02.154803    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:02.367295 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:02.367260    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:02.572308 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:02.572268    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:02.778462 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:02.778421    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:02.983258 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:02.983204    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:03.768638 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:03.768534    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:04.028690 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:04.028608    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:04.301963 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:04.301862    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:04.305628 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:04.305599    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:20:07.982872 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:07.982821    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:09.218363 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.218309    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:09.311464 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.311431    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"]
Nov 04 02:20:09.312127 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.312111    4512 cpu_manager.go:414] "containerMap: removing container" podUID="f6227bc3-6dcb-46e4-b990-07c9ad43252a" containerName="payload-pre-processing"
Nov 04 02:20:09.312171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.312129    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6227bc3-6dcb-46e4-b990-07c9ad43252a" containerName="payload-pre-processing"
Nov 04 02:20:09.312171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.312154    4512 cpu_manager.go:414] "containerMap: removing container" podUID="891a1e83-0481-4fdf-bc83-91c61b411557" containerName="payload-processing"
Nov 04 02:20:09.312171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.312159    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="891a1e83-0481-4fdf-bc83-91c61b411557" containerName="payload-processing"
Nov 04 02:20:09.312171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.312169    4512 cpu_manager.go:414] "containerMap: removing container" podUID="885e2d43-195b-4be3-8459-99634df20617" containerName="revoke-keys"
Nov 04 02:20:09.312296 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.312174    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="885e2d43-195b-4be3-8459-99634df20617" containerName="revoke-keys"
Nov 04 02:20:09.312296 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.312274    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6227bc3-6dcb-46e4-b990-07c9ad43252a" containerName="payload-pre-processing"
Nov 04 02:20:09.312296 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.312284    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="891a1e83-0481-4fdf-bc83-91c61b411557" containerName="payload-processing"
Nov 04 02:20:09.312296 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.312291    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="885e2d43-195b-4be3-8459-99634df20617" containerName="revoke-keys"
Nov 04 02:20:09.315940 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.315922    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:09.352221 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.352191    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2npl6\" (UniqueName: \"kubernetes.io/projected/6ac53138-d0d3-4cf0-ae5a-b0040a825734-kube-api-access-2npl6\") pod \"maas-api-revoke-sub-5e78b8ff-fvk4h\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:09.352353 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.352334    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5e78b8ff-fvk4h\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:09.352353 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.352351    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5e78b8ff-fvk4h\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:09.371954 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.371931    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"]
Nov 04 02:20:09.416665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.416631    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:20:09.416800 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.416676    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:20:09.453143 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.453108    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2npl6\" (UniqueName: \"kubernetes.io/projected/6ac53138-d0d3-4cf0-ae5a-b0040a825734-kube-api-access-2npl6\") pod \"maas-api-revoke-sub-5e78b8ff-fvk4h\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:09.453298 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.453254    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5e78b8ff-fvk4h\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:09.453298 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.453275    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5e78b8ff-fvk4h\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:09.454060 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.454035    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5e78b8ff-fvk4h\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:09.455969 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.455946    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5e78b8ff-fvk4h\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:09.514668 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.514628    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2npl6\" (UniqueName: \"kubernetes.io/projected/6ac53138-d0d3-4cf0-ae5a-b0040a825734-kube-api-access-2npl6\") pod \"maas-api-revoke-sub-5e78b8ff-fvk4h\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:09.632720 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:09.632688    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:10.021886 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:20:10.021850    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ac53138_d0d3_4cf0_ae5a_b0040a825734.slice/crio-3c565e8b6c73a69bf21700b09e82872e10afbf3de0f68ddc88047832be1700bd WatchSource:0}: Error finding container 3c565e8b6c73a69bf21700b09e82872e10afbf3de0f68ddc88047832be1700bd: Status 404 returned error can't find the container with id 3c565e8b6c73a69bf21700b09e82872e10afbf3de0f68ddc88047832be1700bd
Nov 04 02:20:10.022009 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:10.021910    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"]
Nov 04 02:20:10.763013 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:10.762980    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h" event={"ID":"6ac53138-d0d3-4cf0-ae5a-b0040a825734","Type":"ContainerStarted","Data":"626aa19f898521702d62e2c12690b75a41c918a73aa2926f5d02fe12afcc0317"}
Nov 04 02:20:10.763471 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:10.763038    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h" event={"ID":"6ac53138-d0d3-4cf0-ae5a-b0040a825734","Type":"ContainerStarted","Data":"3c565e8b6c73a69bf21700b09e82872e10afbf3de0f68ddc88047832be1700bd"}
Nov 04 02:20:10.884077 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:10.884025    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h" podStartSLOduration=1.8840102459999999 podStartE2EDuration="1.884010246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:20:10.881454358 +0000 UTC m=+1472.686911000" watchObservedRunningTime="2026-11-04 02:20:10.884010246 +0000 UTC m=+1472.689466878"
Nov 04 02:20:11.772235 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:11.772199    4512 generic.go:356] "Generic (PLEG): container finished" podID="6ac53138-d0d3-4cf0-ae5a-b0040a825734" containerID="626aa19f898521702d62e2c12690b75a41c918a73aa2926f5d02fe12afcc0317" exitCode=0
Nov 04 02:20:11.772710 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:11.772259    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h" event={"ID":"6ac53138-d0d3-4cf0-ae5a-b0040a825734","Type":"ContainerDied","Data":"626aa19f898521702d62e2c12690b75a41c918a73aa2926f5d02fe12afcc0317"}
Nov 04 02:20:12.956522 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:12.956472    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:12.998037 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:12.998006    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-xks-ca\") pod \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") "
Nov 04 02:20:12.998179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:12.998070    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-openshift-ca\") pod \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") "
Nov 04 02:20:12.998179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:12.998104    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ac53138-d0d3-4cf0-ae5a-b0040a825734-kube-api-access-2npl6\" (UniqueName: \"kubernetes.io/projected/6ac53138-d0d3-4cf0-ae5a-b0040a825734-kube-api-access-2npl6\") pod \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\" (UID: \"6ac53138-d0d3-4cf0-ae5a-b0040a825734\") "
Nov 04 02:20:12.998444 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:12.998422    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-openshift-ca" pod "6ac53138-d0d3-4cf0-ae5a-b0040a825734" (UID: "6ac53138-d0d3-4cf0-ae5a-b0040a825734"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:20:12.998626 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:12.998607    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:13.000447 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:13.000419    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-xks-ca" pod "6ac53138-d0d3-4cf0-ae5a-b0040a825734" (UID: "6ac53138-d0d3-4cf0-ae5a-b0040a825734"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:20:13.000568 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:13.000466    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac53138-d0d3-4cf0-ae5a-b0040a825734-kube-api-access-2npl6" pod "6ac53138-d0d3-4cf0-ae5a-b0040a825734" (UID: "6ac53138-d0d3-4cf0-ae5a-b0040a825734"). InnerVolumeSpecName "kube-api-access-2npl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:20:13.100006 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:13.099929    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ac53138-d0d3-4cf0-ae5a-b0040a825734-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:13.100006 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:13.099958    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2npl6\" (UniqueName: \"kubernetes.io/projected/6ac53138-d0d3-4cf0-ae5a-b0040a825734-kube-api-access-2npl6\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:13.789949 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:13.789921    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"
Nov 04 02:20:13.790117 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:13.789953    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h" event={"ID":"6ac53138-d0d3-4cf0-ae5a-b0040a825734","Type":"ContainerDied","Data":"3c565e8b6c73a69bf21700b09e82872e10afbf3de0f68ddc88047832be1700bd"}
Nov 04 02:20:13.790117 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:13.790000    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c565e8b6c73a69bf21700b09e82872e10afbf3de0f68ddc88047832be1700bd"
Nov 04 02:20:14.074039 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:14.073952    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:14.306062 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:14.306028    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:20:18.891018 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:18.890802    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:22.596559 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:22.595484    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:22.814890 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:22.814851    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:23.023768 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:23.023730    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:23.379151 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:23.379046    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:23.773990 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:23.773952    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:24.306224 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:24.306186    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:20:26.779837 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:20:26.779801    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:20:28.680294 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:28.680257    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:29.522476 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:29.522437    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:29.769352 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:29.769307    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:32.638970 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:32.638933    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:33.590255 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:33.590215    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:34.307184 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:34.307145    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:20:35.924329 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:35.924288    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:36.171850 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:36.171809    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:36.410726 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:36.410692    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:38.473382 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:38.473343    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:38.697267 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:38.697233    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:20:38.703704 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:38.703677    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:20:42.441558 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:42.441524    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:43.089044 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:43.089008    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"]
Nov 04 02:20:43.202846 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:43.202812    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"]
Nov 04 02:20:44.315532 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:44.314605    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:20:44.410035 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:44.409993    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:47.992550 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:47.992500    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:48.090018 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:48.089979    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk" podUID="82e551cd-7dc1-4cfc-b295-2ddfec61963a" containerName="payload-processing" containerID="cri-o://3ee1e468c4e1667381baab20c26d5487c9b0e41f81345f51c1e1fe004378c996" gracePeriod=25
Nov 04 02:20:48.204006 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:48.203955    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp" podUID="0783f96e-35a1-45a3-906e-8511da03229a" containerName="payload-pre-processing" containerID="cri-o://774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76" gracePeriod=25
Nov 04 02:20:48.616651 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:48.616615    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:49.854349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:49.854311    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:54.305764 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:54.305725    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:20:54.465974 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:54.465934    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:54.720818 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:54.720737    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:55.088857 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:55.088818    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp" podUID="0783f96e-35a1-45a3-906e-8511da03229a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:20:55.089054 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:55.088818    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk" podUID="82e551cd-7dc1-4cfc-b295-2ddfec61963a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:20:56.083334 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:56.083263    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:56.609752 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:56.609713    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:56.778068 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:20:56.778029    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:20:57.417788 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:20:57.417748    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:00.025753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:00.025719    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:03.682090 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:03.682052    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:04.008370 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.008329    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"]
Nov 04 02:21:04.009348 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.009325    4512 cpu_manager.go:414] "containerMap: removing container" podUID="6ac53138-d0d3-4cf0-ae5a-b0040a825734" containerName="revoke-keys"
Nov 04 02:21:04.009460 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.009349    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ac53138-d0d3-4cf0-ae5a-b0040a825734" containerName="revoke-keys"
Nov 04 02:21:04.009544 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.009532    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ac53138-d0d3-4cf0-ae5a-b0040a825734" containerName="revoke-keys"
Nov 04 02:21:04.013617 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.013588    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.064186 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.064157    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3dd3c54c-52dsn\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.064339 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.064282    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3dd3c54c-52dsn\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.064339 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.064318    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvgkf\" (UniqueName: \"kubernetes.io/projected/b81dd325-e1d2-4a2f-825d-1228ea1457b8-kube-api-access-fvgkf\") pod \"maas-api-revoke-sub-3dd3c54c-52dsn\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.069881 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.069858    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"]
Nov 04 02:21:04.114561 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.114401    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:21:04.114957 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.114440    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:21:04.166105 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.166079    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3dd3c54c-52dsn\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.166233 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.166110    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvgkf\" (UniqueName: \"kubernetes.io/projected/b81dd325-e1d2-4a2f-825d-1228ea1457b8-kube-api-access-fvgkf\") pod \"maas-api-revoke-sub-3dd3c54c-52dsn\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.166334 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.166315    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3dd3c54c-52dsn\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.166877 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.166848    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3dd3c54c-52dsn\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.168798 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.168779    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3dd3c54c-52dsn\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.227381 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.227338    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvgkf\" (UniqueName: \"kubernetes.io/projected/b81dd325-e1d2-4a2f-825d-1228ea1457b8-kube-api-access-fvgkf\") pod \"maas-api-revoke-sub-3dd3c54c-52dsn\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.306366 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.306270    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:21:04.327495 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.327464    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:04.709028 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.708990    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"]
Nov 04 02:21:04.709629 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:21:04.709596    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb81dd325_e1d2_4a2f_825d_1228ea1457b8.slice/crio-e6d20eab8eac1f6bec9ce82bd81ec49ff5a58aded526892048c13781e52f914c WatchSource:0}: Error finding container e6d20eab8eac1f6bec9ce82bd81ec49ff5a58aded526892048c13781e52f914c: Status 404 returned error can't find the container with id e6d20eab8eac1f6bec9ce82bd81ec49ff5a58aded526892048c13781e52f914c
Nov 04 02:21:04.748488 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.748458    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"]
Nov 04 02:21:04.858972 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:04.858933    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"]
Nov 04 02:21:05.089017 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:05.088972    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp" podUID="0783f96e-35a1-45a3-906e-8511da03229a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:21:05.089017 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:05.089000    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk" podUID="82e551cd-7dc1-4cfc-b295-2ddfec61963a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:21:05.179079 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:05.178581    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:05.231268 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:05.231237    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn" event={"ID":"b81dd325-e1d2-4a2f-825d-1228ea1457b8","Type":"ContainerStarted","Data":"a5a99ea287b65a66b22d1853a9bd1330d3a3c622aa8e21a765ea7bea2cb66c76"}
Nov 04 02:21:05.231454 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:05.231297    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn" event={"ID":"b81dd325-e1d2-4a2f-825d-1228ea1457b8","Type":"ContainerStarted","Data":"e6d20eab8eac1f6bec9ce82bd81ec49ff5a58aded526892048c13781e52f914c"}
Nov 04 02:21:05.346919 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:05.346807    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn" podStartSLOduration=2.346787447 podStartE2EDuration="2.346787447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:05.34632578 +0000 UTC m=+1527.151782412" watchObservedRunningTime="2026-11-04 02:21:05.346787447 +0000 UTC m=+1527.152244079"
Nov 04 02:21:05.379781 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:05.379733    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:06.240749 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:06.240720    4512 generic.go:356] "Generic (PLEG): container finished" podID="b81dd325-e1d2-4a2f-825d-1228ea1457b8" containerID="a5a99ea287b65a66b22d1853a9bd1330d3a3c622aa8e21a765ea7bea2cb66c76" exitCode=0
Nov 04 02:21:06.241201 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:06.240807    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn" event={"ID":"b81dd325-e1d2-4a2f-825d-1228ea1457b8","Type":"ContainerDied","Data":"a5a99ea287b65a66b22d1853a9bd1330d3a3c622aa8e21a765ea7bea2cb66c76"}
Nov 04 02:21:07.399877 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:07.399855    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:07.511751 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:07.511678    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-openshift-ca\") pod \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") "
Nov 04 02:21:07.511905 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:07.511781    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-xks-ca\") pod \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") "
Nov 04 02:21:07.511905 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:07.511822    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b81dd325-e1d2-4a2f-825d-1228ea1457b8-kube-api-access-fvgkf\" (UniqueName: \"kubernetes.io/projected/b81dd325-e1d2-4a2f-825d-1228ea1457b8-kube-api-access-fvgkf\") pod \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\" (UID: \"b81dd325-e1d2-4a2f-825d-1228ea1457b8\") "
Nov 04 02:21:07.512073 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:07.512048    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-openshift-ca" pod "b81dd325-e1d2-4a2f-825d-1228ea1457b8" (UID: "b81dd325-e1d2-4a2f-825d-1228ea1457b8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:07.512216 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:07.512201    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:07.513695 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:07.513672    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-xks-ca" pod "b81dd325-e1d2-4a2f-825d-1228ea1457b8" (UID: "b81dd325-e1d2-4a2f-825d-1228ea1457b8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:07.513788 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:07.513770    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b81dd325-e1d2-4a2f-825d-1228ea1457b8-kube-api-access-fvgkf" pod "b81dd325-e1d2-4a2f-825d-1228ea1457b8" (UID: "b81dd325-e1d2-4a2f-825d-1228ea1457b8"). InnerVolumeSpecName "kube-api-access-fvgkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:07.613317 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:07.613285    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b81dd325-e1d2-4a2f-825d-1228ea1457b8-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:07.613317 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:07.613313    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvgkf\" (UniqueName: \"kubernetes.io/projected/b81dd325-e1d2-4a2f-825d-1228ea1457b8-kube-api-access-fvgkf\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:08.258630 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.258590    4512 generic.go:356] "Generic (PLEG): container finished" podID="82e551cd-7dc1-4cfc-b295-2ddfec61963a" containerID="3ee1e468c4e1667381baab20c26d5487c9b0e41f81345f51c1e1fe004378c996" exitCode=0
Nov 04 02:21:08.258794 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.258662    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk" event={"ID":"82e551cd-7dc1-4cfc-b295-2ddfec61963a","Type":"ContainerDied","Data":"3ee1e468c4e1667381baab20c26d5487c9b0e41f81345f51c1e1fe004378c996"}
Nov 04 02:21:08.260216 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.260196    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"
Nov 04 02:21:08.260317 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.260192    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn" event={"ID":"b81dd325-e1d2-4a2f-825d-1228ea1457b8","Type":"ContainerDied","Data":"e6d20eab8eac1f6bec9ce82bd81ec49ff5a58aded526892048c13781e52f914c"}
Nov 04 02:21:08.260371 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.260318    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6d20eab8eac1f6bec9ce82bd81ec49ff5a58aded526892048c13781e52f914c"
Nov 04 02:21:08.351336 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.351285    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:21:08.433414 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.433372    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82e551cd-7dc1-4cfc-b295-2ddfec61963a-kube-api-access-l7sh9\" (UniqueName: \"kubernetes.io/projected/82e551cd-7dc1-4cfc-b295-2ddfec61963a-kube-api-access-l7sh9\") pod \"82e551cd-7dc1-4cfc-b295-2ddfec61963a\" (UID: \"82e551cd-7dc1-4cfc-b295-2ddfec61963a\") "
Nov 04 02:21:08.434215 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.434193    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82e551cd-7dc1-4cfc-b295-2ddfec61963a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e551cd-7dc1-4cfc-b295-2ddfec61963a-plugins-config-volume\") pod \"82e551cd-7dc1-4cfc-b295-2ddfec61963a\" (UID: \"82e551cd-7dc1-4cfc-b295-2ddfec61963a\") "
Nov 04 02:21:08.435299 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.434717    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82e551cd-7dc1-4cfc-b295-2ddfec61963a-plugins-config-volume" pod "82e551cd-7dc1-4cfc-b295-2ddfec61963a" (UID: "82e551cd-7dc1-4cfc-b295-2ddfec61963a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:08.438567 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.438539    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e551cd-7dc1-4cfc-b295-2ddfec61963a-kube-api-access-l7sh9" pod "82e551cd-7dc1-4cfc-b295-2ddfec61963a" (UID: "82e551cd-7dc1-4cfc-b295-2ddfec61963a"). InnerVolumeSpecName "kube-api-access-l7sh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:08.520659 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.520632    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:21:08.536019 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.535990    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e551cd-7dc1-4cfc-b295-2ddfec61963a-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:08.536019 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.536014    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7sh9\" (UniqueName: \"kubernetes.io/projected/82e551cd-7dc1-4cfc-b295-2ddfec61963a-kube-api-access-l7sh9\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:08.636743 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.636659    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0783f96e-35a1-45a3-906e-8511da03229a-kube-api-access-q4v8x\" (UniqueName: \"kubernetes.io/projected/0783f96e-35a1-45a3-906e-8511da03229a-kube-api-access-q4v8x\") pod \"0783f96e-35a1-45a3-906e-8511da03229a\" (UID: \"0783f96e-35a1-45a3-906e-8511da03229a\") "
Nov 04 02:21:08.636874 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.636778    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0783f96e-35a1-45a3-906e-8511da03229a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0783f96e-35a1-45a3-906e-8511da03229a-plugins-config-volume\") pod \"0783f96e-35a1-45a3-906e-8511da03229a\" (UID: \"0783f96e-35a1-45a3-906e-8511da03229a\") "
Nov 04 02:21:08.637417 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.637354    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0783f96e-35a1-45a3-906e-8511da03229a-plugins-config-volume" pod "0783f96e-35a1-45a3-906e-8511da03229a" (UID: "0783f96e-35a1-45a3-906e-8511da03229a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:08.639185 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.639154    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0783f96e-35a1-45a3-906e-8511da03229a-kube-api-access-q4v8x" pod "0783f96e-35a1-45a3-906e-8511da03229a" (UID: "0783f96e-35a1-45a3-906e-8511da03229a"). InnerVolumeSpecName "kube-api-access-q4v8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:08.738494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.738466    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0783f96e-35a1-45a3-906e-8511da03229a-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:08.738494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:08.738491    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4v8x\" (UniqueName: \"kubernetes.io/projected/0783f96e-35a1-45a3-906e-8511da03229a-kube-api-access-q4v8x\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:09.269696 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.269656    4512 generic.go:356] "Generic (PLEG): container finished" podID="0783f96e-35a1-45a3-906e-8511da03229a" containerID="774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76" exitCode=0
Nov 04 02:21:09.269696 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.269690    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"
Nov 04 02:21:09.269954 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.269738    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp" event={"ID":"0783f96e-35a1-45a3-906e-8511da03229a","Type":"ContainerDied","Data":"774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76"}
Nov 04 02:21:09.269954 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.269795    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp" event={"ID":"0783f96e-35a1-45a3-906e-8511da03229a","Type":"ContainerDied","Data":"d7c474dda9402e12535ad0fbe3ade8b7f14dc00a2a0a8deb1af2bee1fe2e96e6"}
Nov 04 02:21:09.269954 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.269823    4512 scope.go:122] "RemoveContainer" containerID="774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76"
Nov 04 02:21:09.271703 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.271675    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk" event={"ID":"82e551cd-7dc1-4cfc-b295-2ddfec61963a","Type":"ContainerDied","Data":"16ca56f42952030bd808e4e37f7f9490c18535219a1ca600779893ce6ae758ce"}
Nov 04 02:21:09.271829 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.271758    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"
Nov 04 02:21:09.283577 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.283554    4512 scope.go:122] "RemoveContainer" containerID="774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76"
Nov 04 02:21:09.283878 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:21:09.283855    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76\": container with ID starting with 774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76 not found: ID does not exist" containerID="774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76"
Nov 04 02:21:09.283938 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.283887    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76"} err="failed to get container status \"774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76\": rpc error: code = NotFound desc = could not find container \"774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76\": container with ID starting with 774ed582a96959d1678bbae3862d3a6b2486b8a1bbefe853f161981034338a76 not found: ID does not exist"
Nov 04 02:21:09.283938 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.283908    4512 scope.go:122] "RemoveContainer" containerID="3ee1e468c4e1667381baab20c26d5487c9b0e41f81345f51c1e1fe004378c996"
Nov 04 02:21:09.436418 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.436349    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"]
Nov 04 02:21:09.441062 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.441035    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk"]
Nov 04 02:21:09.504796 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.504755    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:09.550864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.550772    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"]
Nov 04 02:21:09.555123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.555099    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp"]
Nov 04 02:21:09.748885 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.748842    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq" podUID="a1a8c919-a691-4100-8472-cab6a495f733" containerName="payload-processing" containerID="cri-o://3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1" gracePeriod=25
Nov 04 02:21:09.859520 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:09.859421    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv" podUID="1d0189f7-fe9e-44a5-bcd5-24f125a267ff" containerName="payload-pre-processing" containerID="cri-o://c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496" gracePeriod=25
Nov 04 02:21:10.033098 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.033072    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:21:10.133082 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.133055    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:21:10.156845 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.156815    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1a8c919-a691-4100-8472-cab6a495f733-kube-api-access-pfvjx\" (UniqueName: \"kubernetes.io/projected/a1a8c919-a691-4100-8472-cab6a495f733-kube-api-access-pfvjx\") pod \"a1a8c919-a691-4100-8472-cab6a495f733\" (UID: \"a1a8c919-a691-4100-8472-cab6a495f733\") "
Nov 04 02:21:10.157055 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.157037    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1a8c919-a691-4100-8472-cab6a495f733-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a8c919-a691-4100-8472-cab6a495f733-plugins-config-volume\") pod \"a1a8c919-a691-4100-8472-cab6a495f733\" (UID: \"a1a8c919-a691-4100-8472-cab6a495f733\") "
Nov 04 02:21:10.157431 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.157405    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1a8c919-a691-4100-8472-cab6a495f733-plugins-config-volume" pod "a1a8c919-a691-4100-8472-cab6a495f733" (UID: "a1a8c919-a691-4100-8472-cab6a495f733"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:10.157565 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.157537    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a8c919-a691-4100-8472-cab6a495f733-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:10.159097 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.159073    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1a8c919-a691-4100-8472-cab6a495f733-kube-api-access-pfvjx" pod "a1a8c919-a691-4100-8472-cab6a495f733" (UID: "a1a8c919-a691-4100-8472-cab6a495f733"). InnerVolumeSpecName "kube-api-access-pfvjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:10.258340 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.258307    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-plugins-config-volume\") pod \"1d0189f7-fe9e-44a5-bcd5-24f125a267ff\" (UID: \"1d0189f7-fe9e-44a5-bcd5-24f125a267ff\") "
Nov 04 02:21:10.258533 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.258372    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-kube-api-access-vx7nk\" (UniqueName: \"kubernetes.io/projected/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-kube-api-access-vx7nk\") pod \"1d0189f7-fe9e-44a5-bcd5-24f125a267ff\" (UID: \"1d0189f7-fe9e-44a5-bcd5-24f125a267ff\") "
Nov 04 02:21:10.258742 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.258718    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-plugins-config-volume" pod "1d0189f7-fe9e-44a5-bcd5-24f125a267ff" (UID: "1d0189f7-fe9e-44a5-bcd5-24f125a267ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:10.258806 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.258775    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfvjx\" (UniqueName: \"kubernetes.io/projected/a1a8c919-a691-4100-8472-cab6a495f733-kube-api-access-pfvjx\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:10.260482 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.260462    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-kube-api-access-vx7nk" pod "1d0189f7-fe9e-44a5-bcd5-24f125a267ff" (UID: "1d0189f7-fe9e-44a5-bcd5-24f125a267ff"). InnerVolumeSpecName "kube-api-access-vx7nk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:10.284375 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.284334    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:10.285894 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.285870    4512 generic.go:356] "Generic (PLEG): container finished" podID="1d0189f7-fe9e-44a5-bcd5-24f125a267ff" containerID="c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496" exitCode=0
Nov 04 02:21:10.286014 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.286004    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv" event={"ID":"1d0189f7-fe9e-44a5-bcd5-24f125a267ff","Type":"ContainerDied","Data":"c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496"}
Nov 04 02:21:10.286054 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.286037    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv" event={"ID":"1d0189f7-fe9e-44a5-bcd5-24f125a267ff","Type":"ContainerDied","Data":"72a33662daa40a1a66e2ccd2b295431cdd18990a459b1f4353027ea5aea5195e"}
Nov 04 02:21:10.286087 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.286059    4512 scope.go:122] "RemoveContainer" containerID="c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496"
Nov 04 02:21:10.286137 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.286130    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"
Nov 04 02:21:10.293572 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.293547    4512 generic.go:356] "Generic (PLEG): container finished" podID="a1a8c919-a691-4100-8472-cab6a495f733" containerID="3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1" exitCode=0
Nov 04 02:21:10.293744 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.293604    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq" event={"ID":"a1a8c919-a691-4100-8472-cab6a495f733","Type":"ContainerDied","Data":"3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1"}
Nov 04 02:21:10.293744 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.293659    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq" event={"ID":"a1a8c919-a691-4100-8472-cab6a495f733","Type":"ContainerDied","Data":"0ebf53b6dc3582d982025340b45375cfc652240282b274a3b7d5c6a6bf60f924"}
Nov 04 02:21:10.293744 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.293696    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"
Nov 04 02:21:10.321188 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.321159    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0783f96e-35a1-45a3-906e-8511da03229a" path="/var/lib/kubelet/pods/0783f96e-35a1-45a3-906e-8511da03229a/volumes"
Nov 04 02:21:10.321700 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.321676    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82e551cd-7dc1-4cfc-b295-2ddfec61963a" path="/var/lib/kubelet/pods/82e551cd-7dc1-4cfc-b295-2ddfec61963a/volumes"
Nov 04 02:21:10.322565 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.322544    4512 scope.go:122] "RemoveContainer" containerID="c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496"
Nov 04 02:21:10.322888 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:21:10.322869    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496\": container with ID starting with c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496 not found: ID does not exist" containerID="c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496"
Nov 04 02:21:10.322978 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.322916    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496"} err="failed to get container status \"c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496\": rpc error: code = NotFound desc = could not find container \"c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496\": container with ID starting with c705fa72976f6f831c1d183fb065fc1a8594bf50096aeb2d89ed9155e759a496 not found: ID does not exist"
Nov 04 02:21:10.322978 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.322944    4512 scope.go:122] "RemoveContainer" containerID="3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1"
Nov 04 02:21:10.341292 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.341263    4512 scope.go:122] "RemoveContainer" containerID="3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1"
Nov 04 02:21:10.341652 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:21:10.341623    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1\": container with ID starting with 3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1 not found: ID does not exist" containerID="3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1"
Nov 04 02:21:10.341737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.341655    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1"} err="failed to get container status \"3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1\": rpc error: code = NotFound desc = could not find container \"3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1\": container with ID starting with 3caf2201c002fd257884b6fd1560c6d79f478b4dfebe9ff5f3a79cada313d1e1 not found: ID does not exist"
Nov 04 02:21:10.360239 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.360213    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:10.360239 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.360238    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx7nk\" (UniqueName: \"kubernetes.io/projected/1d0189f7-fe9e-44a5-bcd5-24f125a267ff-kube-api-access-vx7nk\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:10.457551 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.457359    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"]
Nov 04 02:21:10.461176 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.461154    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv"]
Nov 04 02:21:10.569572 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.569540    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"]
Nov 04 02:21:10.574382 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:10.574355    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq"]
Nov 04 02:21:12.313928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:12.313892    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d0189f7-fe9e-44a5-bcd5-24f125a267ff" path="/var/lib/kubelet/pods/1d0189f7-fe9e-44a5-bcd5-24f125a267ff/volumes"
Nov 04 02:21:12.314410 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:12.314318    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1a8c919-a691-4100-8472-cab6a495f733" path="/var/lib/kubelet/pods/a1a8c919-a691-4100-8472-cab6a495f733/volumes"
Nov 04 02:21:14.305974 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:14.305928    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:21:15.186202 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:15.186148    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:17.351284 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:17.351251    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:17.562299 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:17.562261    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:17.765147 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:17.765118    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:18.098849 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:18.098748    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:20.086962 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:20.086922    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:21.128932 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:21.128900    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"]
Nov 04 02:21:21.129369 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:21.129123    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerName="istio-proxy" containerID="cri-o://894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876" gracePeriod=30
Nov 04 02:21:24.119311 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:24.119271    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:24.306406 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:24.306370    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:21:24.408412 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:24.408314    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:24.425087 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:24.425049    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:21:24.425272 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:24.425122    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:21:25.091453 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:25.091399    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:26.397283 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.397214    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:21:26.416135 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.416102    4512 generic.go:356] "Generic (PLEG): container finished" podID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerID="894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876" exitCode=0
Nov 04 02:21:26.416280 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.416161    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"
Nov 04 02:21:26.416280 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.416187    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" event={"ID":"3c1f6ba3-41f4-4d64-8873-6642f101d280","Type":"ContainerDied","Data":"894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876"}
Nov 04 02:21:26.416280 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.416239    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd" event={"ID":"3c1f6ba3-41f4-4d64-8873-6642f101d280","Type":"ContainerDied","Data":"37710b81f99ce305e56766c1355eecf0ffdb6928926243aace34debd1cb63fe4"}
Nov 04 02:21:26.416280 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.416264    4512 scope.go:122] "RemoveContainer" containerID="894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876"
Nov 04 02:21:26.428964 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.428939    4512 scope.go:122] "RemoveContainer" containerID="894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876"
Nov 04 02:21:26.429262 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:21:26.429242    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876\": container with ID starting with 894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876 not found: ID does not exist" containerID="894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876"
Nov 04 02:21:26.429338 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.429269    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876"} err="failed to get container status \"894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876\": rpc error: code = NotFound desc = could not find container \"894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876\": container with ID starting with 894a233c1d4be53504f01fb909a55b6210c962ecb0e2c9f4cfd4c7587f9b0876 not found: ID does not exist"
Nov 04 02:21:26.469346 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469315    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-certs\") pod \"3c1f6ba3-41f4-4d64-8873-6642f101d280\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") "
Nov 04 02:21:26.469537 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469379    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-kube-api-access-djdf8\" (UniqueName: \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-kube-api-access-djdf8\") pod \"3c1f6ba3-41f4-4d64-8873-6642f101d280\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") "
Nov 04 02:21:26.469537 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469436    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-socket\") pod \"3c1f6ba3-41f4-4d64-8873-6642f101d280\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") "
Nov 04 02:21:26.469537 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469480    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-podinfo\") pod \"3c1f6ba3-41f4-4d64-8873-6642f101d280\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") "
Nov 04 02:21:26.469818 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469548    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-data\") pod \"3c1f6ba3-41f4-4d64-8873-6642f101d280\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") "
Nov 04 02:21:26.469818 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469588    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c1f6ba3-41f4-4d64-8873-6642f101d280-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c1f6ba3-41f4-4d64-8873-6642f101d280-istiod-ca-cert\") pod \"3c1f6ba3-41f4-4d64-8873-6642f101d280\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") "
Nov 04 02:21:26.469818 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469613    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-envoy\") pod \"3c1f6ba3-41f4-4d64-8873-6642f101d280\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") "
Nov 04 02:21:26.469818 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469623    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-certs" pod "3c1f6ba3-41f4-4d64-8873-6642f101d280" (UID: "3c1f6ba3-41f4-4d64-8873-6642f101d280"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:26.469818 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469713    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-token\" (UniqueName: \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-token\") pod \"3c1f6ba3-41f4-4d64-8873-6642f101d280\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") "
Nov 04 02:21:26.469818 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469773    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-socket" pod "3c1f6ba3-41f4-4d64-8873-6642f101d280" (UID: "3c1f6ba3-41f4-4d64-8873-6642f101d280"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:26.470117 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.469853    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-credential-socket\") pod \"3c1f6ba3-41f4-4d64-8873-6642f101d280\" (UID: \"3c1f6ba3-41f4-4d64-8873-6642f101d280\") "
Nov 04 02:21:26.470170 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.470137    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-credential-socket" pod "3c1f6ba3-41f4-4d64-8873-6642f101d280" (UID: "3c1f6ba3-41f4-4d64-8873-6642f101d280"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:26.470629 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.470571    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-data" pod "3c1f6ba3-41f4-4d64-8873-6642f101d280" (UID: "3c1f6ba3-41f4-4d64-8873-6642f101d280"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:26.470726 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.470677    4512 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:26.470726 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.470699    4512 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-credential-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:26.470726 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.470715    4512 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-workload-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:26.470726 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.470676    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c1f6ba3-41f4-4d64-8873-6642f101d280-istiod-ca-cert" pod "3c1f6ba3-41f4-4d64-8873-6642f101d280" (UID: "3c1f6ba3-41f4-4d64-8873-6642f101d280"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:26.472195 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.472158    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-token" pod "3c1f6ba3-41f4-4d64-8873-6642f101d280" (UID: "3c1f6ba3-41f4-4d64-8873-6642f101d280"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:26.472298 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.472207    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-kube-api-access-djdf8" pod "3c1f6ba3-41f4-4d64-8873-6642f101d280" (UID: "3c1f6ba3-41f4-4d64-8873-6642f101d280"). InnerVolumeSpecName "kube-api-access-djdf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:26.472620 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.472595    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-envoy" pod "3c1f6ba3-41f4-4d64-8873-6642f101d280" (UID: "3c1f6ba3-41f4-4d64-8873-6642f101d280"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:26.472719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.472678    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-podinfo" pod "3c1f6ba3-41f4-4d64-8873-6642f101d280" (UID: "3c1f6ba3-41f4-4d64-8873-6642f101d280"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:21:26.572002 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.571966    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djdf8\" (UniqueName: \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-kube-api-access-djdf8\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:26.572002 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.572002    4512 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-podinfo\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:26.572002 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.572016    4512 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-data\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:26.572292 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.572030    4512 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c1f6ba3-41f4-4d64-8873-6642f101d280-istiod-ca-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:26.572292 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.572044    4512 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-envoy\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:26.572292 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.572057    4512 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c1f6ba3-41f4-4d64-8873-6642f101d280-istio-token\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:26.779250 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:21:26.779214    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:21:26.846425 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.846383    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"]
Nov 04 02:21:26.850151 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:26.850119    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd"]
Nov 04 02:21:28.317613 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:28.317579    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" path="/var/lib/kubelet/pods/3c1f6ba3-41f4-4d64-8873-6642f101d280/volumes"
Nov 04 02:21:29.358157 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:29.358119    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:30.306723 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:30.306685    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:21:30.312747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:30.312710    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:30.560979 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:30.560880    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:31.100703 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:31.100659    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:31.560069 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:31.560030    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:31.776371 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:31.776321    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:32.990434 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:32.990391    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:33.306987 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:33.306887    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:33.510141 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:33.510100    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:33.597855 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:33.597771    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"]
Nov 04 02:21:33.599266 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:33.599227    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:21:35.038770 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:35.038730    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:35.515597 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:35.515491    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:35.999540 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:35.999490    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:36.204725 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.204682    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:36.556330 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.556290    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"]
Nov 04 02:21:36.557149 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557133    4512 cpu_manager.go:414] "containerMap: removing container" podUID="0783f96e-35a1-45a3-906e-8511da03229a" containerName="payload-pre-processing"
Nov 04 02:21:36.557198 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557153    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0783f96e-35a1-45a3-906e-8511da03229a" containerName="payload-pre-processing"
Nov 04 02:21:36.557198 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557176    4512 cpu_manager.go:414] "containerMap: removing container" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerName="istio-proxy"
Nov 04 02:21:36.557198 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557183    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerName="istio-proxy"
Nov 04 02:21:36.557287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557198    4512 cpu_manager.go:414] "containerMap: removing container" podUID="a1a8c919-a691-4100-8472-cab6a495f733" containerName="payload-processing"
Nov 04 02:21:36.557287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557205    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1a8c919-a691-4100-8472-cab6a495f733" containerName="payload-processing"
Nov 04 02:21:36.557287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557229    4512 cpu_manager.go:414] "containerMap: removing container" podUID="82e551cd-7dc1-4cfc-b295-2ddfec61963a" containerName="payload-processing"
Nov 04 02:21:36.557287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557236    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82e551cd-7dc1-4cfc-b295-2ddfec61963a" containerName="payload-processing"
Nov 04 02:21:36.557287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557248    4512 cpu_manager.go:414] "containerMap: removing container" podUID="b81dd325-e1d2-4a2f-825d-1228ea1457b8" containerName="revoke-keys"
Nov 04 02:21:36.557287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557254    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b81dd325-e1d2-4a2f-825d-1228ea1457b8" containerName="revoke-keys"
Nov 04 02:21:36.557287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557278    4512 cpu_manager.go:414] "containerMap: removing container" podUID="1d0189f7-fe9e-44a5-bcd5-24f125a267ff" containerName="payload-pre-processing"
Nov 04 02:21:36.557287 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557285    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d0189f7-fe9e-44a5-bcd5-24f125a267ff" containerName="payload-pre-processing"
Nov 04 02:21:36.557534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557418    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="82e551cd-7dc1-4cfc-b295-2ddfec61963a" containerName="payload-processing"
Nov 04 02:21:36.557534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557428    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="b81dd325-e1d2-4a2f-825d-1228ea1457b8" containerName="revoke-keys"
Nov 04 02:21:36.557534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557442    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d0189f7-fe9e-44a5-bcd5-24f125a267ff" containerName="payload-pre-processing"
Nov 04 02:21:36.557534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557455    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="0783f96e-35a1-45a3-906e-8511da03229a" containerName="payload-pre-processing"
Nov 04 02:21:36.557534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557464    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c1f6ba3-41f4-4d64-8873-6642f101d280" containerName="istio-proxy"
Nov 04 02:21:36.557534 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.557473    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1a8c919-a691-4100-8472-cab6a495f733" containerName="payload-processing"
Nov 04 02:21:36.561530 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.561500    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:36.616832 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.616806    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"]
Nov 04 02:21:36.641136 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.641101    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-80f643e6-fp4xt\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:36.641261 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.641171    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkgbz\" (UniqueName: \"kubernetes.io/projected/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-kube-api-access-kkgbz\") pod \"maas-api-revoke-sub-80f643e6-fp4xt\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:36.641322 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.641299    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-80f643e6-fp4xt\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:36.661078 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.661053    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:21:36.661213 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.661174    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:21:36.742877 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.742780    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-80f643e6-fp4xt\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:36.743012 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.742979    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkgbz\" (UniqueName: \"kubernetes.io/projected/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-kube-api-access-kkgbz\") pod \"maas-api-revoke-sub-80f643e6-fp4xt\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:36.743164 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.743057    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-80f643e6-fp4xt\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:36.743457 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.743434    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-80f643e6-fp4xt\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:36.745490 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.745470    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-80f643e6-fp4xt\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:36.804459 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.804427    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkgbz\" (UniqueName: \"kubernetes.io/projected/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-kube-api-access-kkgbz\") pod \"maas-api-revoke-sub-80f643e6-fp4xt\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:36.875942 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:36.875854    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:37.257805 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:37.257773    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"]
Nov 04 02:21:37.258942 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:21:37.258910    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podede5ce9b_38db_40d8_bf49_c9b645fbbbfb.slice/crio-65880f84145ba57de7a57b594eb1cff90c076974964da3606dd6796abd4f4024 WatchSource:0}: Error finding container 65880f84145ba57de7a57b594eb1cff90c076974964da3606dd6796abd4f4024: Status 404 returned error can't find the container with id 65880f84145ba57de7a57b594eb1cff90c076974964da3606dd6796abd4f4024
Nov 04 02:21:37.499230 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:37.499191    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt" event={"ID":"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb","Type":"ContainerStarted","Data":"65880f84145ba57de7a57b594eb1cff90c076974964da3606dd6796abd4f4024"}
Nov 04 02:21:38.511671 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.511630    4512 generic.go:356] "Generic (PLEG): container finished" podID="ede5ce9b-38db-40d8-bf49-c9b645fbbbfb" containerID="8efe81873214c9d57bebd63a6b68cb4ac707752c903993ef92532d1075c24b96" exitCode=0
Nov 04 02:21:38.512039 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.511681    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt" event={"ID":"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb","Type":"ContainerDied","Data":"8efe81873214c9d57bebd63a6b68cb4ac707752c903993ef92532d1075c24b96"}
Nov 04 02:21:38.599038 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.598925    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing" containerID="cri-o://d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c" gracePeriod=25
Nov 04 02:21:38.870005 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.869939    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:21:38.971268 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.971228    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-provider-credentials\" (UniqueName: \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-provider-credentials\") pod \"2704e4da-d3aa-4c9c-bf29-c677270630c5\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") "
Nov 04 02:21:38.971527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.971297    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-kube-api-access-55l8t\" (UniqueName: \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-kube-api-access-55l8t\") pod \"2704e4da-d3aa-4c9c-bf29-c677270630c5\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") "
Nov 04 02:21:38.971527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.971377    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-routing-overlay-volume\") pod \"2704e4da-d3aa-4c9c-bf29-c677270630c5\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") "
Nov 04 02:21:38.971527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.971454    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-plugins-config-volume\") pod \"2704e4da-d3aa-4c9c-bf29-c677270630c5\" (UID: \"2704e4da-d3aa-4c9c-bf29-c677270630c5\") "
Nov 04 02:21:38.972092 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.972008    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-routing-overlay-volume" pod "2704e4da-d3aa-4c9c-bf29-c677270630c5" (UID: "2704e4da-d3aa-4c9c-bf29-c677270630c5"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:38.972361 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.972333    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-plugins-config-volume" pod "2704e4da-d3aa-4c9c-bf29-c677270630c5" (UID: "2704e4da-d3aa-4c9c-bf29-c677270630c5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:38.973900 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.973873    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-kube-api-access-55l8t" pod "2704e4da-d3aa-4c9c-bf29-c677270630c5" (UID: "2704e4da-d3aa-4c9c-bf29-c677270630c5"). InnerVolumeSpecName "kube-api-access-55l8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:38.973994 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:38.973896    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-provider-credentials" pod "2704e4da-d3aa-4c9c-bf29-c677270630c5" (UID: "2704e4da-d3aa-4c9c-bf29-c677270630c5"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:39.073272 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.073182    4512 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-provider-credentials\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:39.073272 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.073270    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55l8t\" (UniqueName: \"kubernetes.io/projected/2704e4da-d3aa-4c9c-bf29-c677270630c5-kube-api-access-55l8t\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:39.073481 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.073285    4512 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-routing-overlay-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:39.073481 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.073293    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2704e4da-d3aa-4c9c-bf29-c677270630c5-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:39.521953 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.521920    4512 generic.go:356] "Generic (PLEG): container finished" podID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerID="d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c" exitCode=0
Nov 04 02:21:39.522452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.522006    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"
Nov 04 02:21:39.522452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.522007    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" event={"ID":"2704e4da-d3aa-4c9c-bf29-c677270630c5","Type":"ContainerDied","Data":"d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c"}
Nov 04 02:21:39.522452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.522083    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w" event={"ID":"2704e4da-d3aa-4c9c-bf29-c677270630c5","Type":"ContainerDied","Data":"39f56361cbd75f0f5fe9ca223308051a66148b4c9a1a3a9bb3afaeafba3d5aa4"}
Nov 04 02:21:39.522452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.522116    4512 scope.go:122] "RemoveContainer" containerID="d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c"
Nov 04 02:21:39.541625 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.541499    4512 scope.go:122] "RemoveContainer" containerID="d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c"
Nov 04 02:21:39.542077 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:21:39.542047    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c\": container with ID starting with d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c not found: ID does not exist" containerID="d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c"
Nov 04 02:21:39.542175 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.542085    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c"} err="failed to get container status \"d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c\": rpc error: code = NotFound desc = could not find container \"d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c\": container with ID starting with d4698041f014934bd38489b1c1b1e705baf24cb3c741c6a6a6a691b09f138b8c not found: ID does not exist"
Nov 04 02:21:39.682782 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.682757    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:39.688872 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.688836    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"]
Nov 04 02:21:39.738165 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.738127    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w"]
Nov 04 02:21:39.781369 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.781275    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-openshift-ca\") pod \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") "
Nov 04 02:21:39.781587 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.781394    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-kube-api-access-kkgbz\" (UniqueName: \"kubernetes.io/projected/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-kube-api-access-kkgbz\") pod \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") "
Nov 04 02:21:39.781587 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.781422    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-xks-ca\") pod \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\" (UID: \"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb\") "
Nov 04 02:21:39.781752 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.781724    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-openshift-ca" pod "ede5ce9b-38db-40d8-bf49-c9b645fbbbfb" (UID: "ede5ce9b-38db-40d8-bf49-c9b645fbbbfb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:39.782054 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.782029    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:39.783742 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.783719    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-kube-api-access-kkgbz" pod "ede5ce9b-38db-40d8-bf49-c9b645fbbbfb" (UID: "ede5ce9b-38db-40d8-bf49-c9b645fbbbfb"). InnerVolumeSpecName "kube-api-access-kkgbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:39.783841 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.783743    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-xks-ca" pod "ede5ce9b-38db-40d8-bf49-c9b645fbbbfb" (UID: "ede5ce9b-38db-40d8-bf49-c9b645fbbbfb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:39.883114 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.883066    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkgbz\" (UniqueName: \"kubernetes.io/projected/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-kube-api-access-kkgbz\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:39.883114 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:39.883106    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:40.315452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:40.315415    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" path="/var/lib/kubelet/pods/2704e4da-d3aa-4c9c-bf29-c677270630c5/volumes"
Nov 04 02:21:40.534623 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:40.534573    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt" event={"ID":"ede5ce9b-38db-40d8-bf49-c9b645fbbbfb","Type":"ContainerDied","Data":"65880f84145ba57de7a57b594eb1cff90c076974964da3606dd6796abd4f4024"}
Nov 04 02:21:40.535060 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:40.534638    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65880f84145ba57de7a57b594eb1cff90c076974964da3606dd6796abd4f4024"
Nov 04 02:21:40.535060 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:40.534589    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"
Nov 04 02:21:40.903778 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:40.903740    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"]
Nov 04 02:21:41.119046 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:41.119007    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:41.378558 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:41.377808    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:45.686405 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:45.686373    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"]
Nov 04 02:21:46.775143 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:46.775106    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:47.237770 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:47.237734    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:50.686766 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.686664    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4" podUID="1d3300f6-b801-481d-a7f9-b76114860e85" containerName="payload-processing" containerID="cri-o://7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe" gracePeriod=25
Nov 04 02:21:50.952233 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.952169    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:21:50.995206 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.995177    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-routing-overlay-volume\") pod \"1d3300f6-b801-481d-a7f9-b76114860e85\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") "
Nov 04 02:21:50.995368 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.995279    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-plugins-config-volume\") pod \"1d3300f6-b801-481d-a7f9-b76114860e85\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") "
Nov 04 02:21:50.995368 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.995305    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-kube-api-access-mfbh2\" (UniqueName: \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-kube-api-access-mfbh2\") pod \"1d3300f6-b801-481d-a7f9-b76114860e85\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") "
Nov 04 02:21:50.995492 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.995422    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials\" (UniqueName: \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials\") pod \"1d3300f6-b801-481d-a7f9-b76114860e85\" (UID: \"1d3300f6-b801-481d-a7f9-b76114860e85\") "
Nov 04 02:21:50.995680 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.995651    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-routing-overlay-volume" pod "1d3300f6-b801-481d-a7f9-b76114860e85" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:50.995767 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.995738    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-plugins-config-volume" pod "1d3300f6-b801-481d-a7f9-b76114860e85" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:50.995967 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.995950    4512 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-routing-overlay-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:50.996024 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.995971    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d3300f6-b801-481d-a7f9-b76114860e85-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:50.998324 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.998292    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials" pod "1d3300f6-b801-481d-a7f9-b76114860e85" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:50.998430 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:50.998382    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-kube-api-access-mfbh2" pod "1d3300f6-b801-481d-a7f9-b76114860e85" (UID: "1d3300f6-b801-481d-a7f9-b76114860e85"). InnerVolumeSpecName "kube-api-access-mfbh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:51.097695 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.097648    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfbh2\" (UniqueName: \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-kube-api-access-mfbh2\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:51.097695 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.097688    4512 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1d3300f6-b801-481d-a7f9-b76114860e85-provider-credentials\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:51.379995 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.379957    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:51.651066 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.650971    4512 generic.go:356] "Generic (PLEG): container finished" podID="1d3300f6-b801-481d-a7f9-b76114860e85" containerID="7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe" exitCode=0
Nov 04 02:21:51.651066 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.651022    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"
Nov 04 02:21:51.651307 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.651060    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4" event={"ID":"1d3300f6-b801-481d-a7f9-b76114860e85","Type":"ContainerDied","Data":"7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe"}
Nov 04 02:21:51.651307 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.651127    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4" event={"ID":"1d3300f6-b801-481d-a7f9-b76114860e85","Type":"ContainerDied","Data":"b0efc945f58c280a6773a2457aed0f4ec313768b2affe131a993404c16ccc73f"}
Nov 04 02:21:51.651307 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.651160    4512 scope.go:122] "RemoveContainer" containerID="7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe"
Nov 04 02:21:51.662616 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.662595    4512 scope.go:122] "RemoveContainer" containerID="7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe"
Nov 04 02:21:51.662908 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:21:51.662883    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe\": container with ID starting with 7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe not found: ID does not exist" containerID="7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe"
Nov 04 02:21:51.663007 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.662914    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe"} err="failed to get container status \"7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe\": rpc error: code = NotFound desc = could not find container \"7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe\": container with ID starting with 7fd6bcdaf4fcc13f71ce60b0f3609c13343597c1ed5abae03fa4282ab50caffe not found: ID does not exist"
Nov 04 02:21:51.820075 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.820038    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"]
Nov 04 02:21:51.869618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:51.869588    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4"]
Nov 04 02:21:52.312552 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:52.312496    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d3300f6-b801-481d-a7f9-b76114860e85" path="/var/lib/kubelet/pods/1d3300f6-b801-481d-a7f9-b76114860e85/volumes"
Nov 04 02:21:52.979641 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:52.979601    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"]
Nov 04 02:21:53.061071 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.061036    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"]
Nov 04 02:21:53.061735 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.061716    4512 cpu_manager.go:414] "containerMap: removing container" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing"
Nov 04 02:21:53.061735 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.061736    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing"
Nov 04 02:21:53.061842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.061748    4512 cpu_manager.go:414] "containerMap: removing container" podUID="1d3300f6-b801-481d-a7f9-b76114860e85" containerName="payload-processing"
Nov 04 02:21:53.061842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.061753    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d3300f6-b801-481d-a7f9-b76114860e85" containerName="payload-processing"
Nov 04 02:21:53.061842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.061777    4512 cpu_manager.go:414] "containerMap: removing container" podUID="ede5ce9b-38db-40d8-bf49-c9b645fbbbfb" containerName="revoke-keys"
Nov 04 02:21:53.061842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.061783    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ede5ce9b-38db-40d8-bf49-c9b645fbbbfb" containerName="revoke-keys"
Nov 04 02:21:53.062014 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.061904    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="2704e4da-d3aa-4c9c-bf29-c677270630c5" containerName="payload-processing"
Nov 04 02:21:53.062014 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.061914    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d3300f6-b801-481d-a7f9-b76114860e85" containerName="payload-processing"
Nov 04 02:21:53.062014 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.061938    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="ede5ce9b-38db-40d8-bf49-c9b645fbbbfb" containerName="revoke-keys"
Nov 04 02:21:53.067163 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.067108    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.085977 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.085950    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"]
Nov 04 02:21:53.119022 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.118985    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5d4fd4a7-hw8tj\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.119692 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.119657    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5d4fd4a7-hw8tj\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.119981 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.119917    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrljg\" (UniqueName: \"kubernetes.io/projected/689d222a-ebe1-4493-b3ef-08c76a8678b3-kube-api-access-jrljg\") pod \"maas-api-revoke-sub-5d4fd4a7-hw8tj\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.121649 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.121627    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:21:53.121763 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.121642    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:21:53.126410 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.126386    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"]
Nov 04 02:21:53.222128 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.222094    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrljg\" (UniqueName: \"kubernetes.io/projected/689d222a-ebe1-4493-b3ef-08c76a8678b3-kube-api-access-jrljg\") pod \"maas-api-revoke-sub-5d4fd4a7-hw8tj\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.222328 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.222227    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5d4fd4a7-hw8tj\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.222399 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.222360    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5d4fd4a7-hw8tj\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.223241 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.223148    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5d4fd4a7-hw8tj\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.225992 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.225967    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5d4fd4a7-hw8tj\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.284536 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.284485    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrljg\" (UniqueName: \"kubernetes.io/projected/689d222a-ebe1-4493-b3ef-08c76a8678b3-kube-api-access-jrljg\") pod \"maas-api-revoke-sub-5d4fd4a7-hw8tj\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.385527 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.385477    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:53.411084 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.407204    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:53.613204 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.613107    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:53.816283 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.816238    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:53.828205 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:53.828171    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"]
Nov 04 02:21:53.829089 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:21:53.829050    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod689d222a_ebe1_4493_b3ef_08c76a8678b3.slice/crio-f1673de711a4a96b6e119663d0aed139521ce63e43115149d2d7504356a2a742 WatchSource:0}: Error finding container f1673de711a4a96b6e119663d0aed139521ce63e43115149d2d7504356a2a742: Status 404 returned error can't find the container with id f1673de711a4a96b6e119663d0aed139521ce63e43115149d2d7504356a2a742
Nov 04 02:21:54.429693 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.429307    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"]
Nov 04 02:21:54.436718 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.434887    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.496029 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.495984    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"]
Nov 04 02:21:54.560085 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.560048    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2djjt\" (UniqueName: \"kubernetes.io/projected/872308d5-8d98-49c7-8424-eddc7de82b41-kube-api-access-2djjt\") pod \"maas-api-revoke-sub-6c0b3916-blgdv\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.560255 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.560176    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6c0b3916-blgdv\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.560255 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.560240    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6c0b3916-blgdv\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.661351 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.661316    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2djjt\" (UniqueName: \"kubernetes.io/projected/872308d5-8d98-49c7-8424-eddc7de82b41-kube-api-access-2djjt\") pod \"maas-api-revoke-sub-6c0b3916-blgdv\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.661574 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.661418    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6c0b3916-blgdv\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.661574 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.661495    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6c0b3916-blgdv\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.662206 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.662182    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6c0b3916-blgdv\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.664277 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.664254    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6c0b3916-blgdv\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.676631 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.676595    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj" event={"ID":"689d222a-ebe1-4493-b3ef-08c76a8678b3","Type":"ContainerStarted","Data":"0a11a6937d9dfaed6c031fb55c4940ebcab592fd561b2fb8650226e83d130502"}
Nov 04 02:21:54.676798 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.676652    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj" event={"ID":"689d222a-ebe1-4493-b3ef-08c76a8678b3","Type":"ContainerStarted","Data":"f1673de711a4a96b6e119663d0aed139521ce63e43115149d2d7504356a2a742"}
Nov 04 02:21:54.725592 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.725472    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2djjt\" (UniqueName: \"kubernetes.io/projected/872308d5-8d98-49c7-8424-eddc7de82b41-kube-api-access-2djjt\") pod \"maas-api-revoke-sub-6c0b3916-blgdv\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.756496 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.756466    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:54.794699 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:54.794626    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj" podStartSLOduration=1.794609946 podStartE2EDuration="1.794609946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:54.791018239 +0000 UTC m=+1576.596474872" watchObservedRunningTime="2026-11-04 02:21:54.794609946 +0000 UTC m=+1576.600066577"
Nov 04 02:21:55.136786 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:55.136754    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"]
Nov 04 02:21:55.137461 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:21:55.137428    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod872308d5_8d98_49c7_8424_eddc7de82b41.slice/crio-f9e947382e0077ea03e816b3ec6f599dc1a74068f2fa6b5f14cae5064a0c26e7 WatchSource:0}: Error finding container f9e947382e0077ea03e816b3ec6f599dc1a74068f2fa6b5f14cae5064a0c26e7: Status 404 returned error can't find the container with id f9e947382e0077ea03e816b3ec6f599dc1a74068f2fa6b5f14cae5064a0c26e7
Nov 04 02:21:55.685585 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:55.685541    4512 generic.go:356] "Generic (PLEG): container finished" podID="689d222a-ebe1-4493-b3ef-08c76a8678b3" containerID="0a11a6937d9dfaed6c031fb55c4940ebcab592fd561b2fb8650226e83d130502" exitCode=0
Nov 04 02:21:55.686441 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:55.685593    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj" event={"ID":"689d222a-ebe1-4493-b3ef-08c76a8678b3","Type":"ContainerDied","Data":"0a11a6937d9dfaed6c031fb55c4940ebcab592fd561b2fb8650226e83d130502"}
Nov 04 02:21:55.687462 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:55.687438    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv" event={"ID":"872308d5-8d98-49c7-8424-eddc7de82b41","Type":"ContainerStarted","Data":"ff10794673a17e788e7a32ea9302a28775f5fb9bc9247dfeb6e757f1bfcf5789"}
Nov 04 02:21:55.687610 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:55.687484    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv" event={"ID":"872308d5-8d98-49c7-8424-eddc7de82b41","Type":"ContainerStarted","Data":"f9e947382e0077ea03e816b3ec6f599dc1a74068f2fa6b5f14cae5064a0c26e7"}
Nov 04 02:21:55.922917 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:55.922802    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv" podStartSLOduration=1.9227877580000001 podStartE2EDuration="1.922787758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:55.92005207 +0000 UTC m=+1577.725508703" watchObservedRunningTime="2026-11-04 02:21:55.922787758 +0000 UTC m=+1577.728244389"
Nov 04 02:21:55.926736 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:55.926666    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j" podUID="5b44b714-aa08-4392-b4f4-f7fafc232d42" containerName="main" containerID="cri-o://f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252" gracePeriod=45
Nov 04 02:21:56.238966 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.238941    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:21:56.282071 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.282042    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b44b714-aa08-4392-b4f4-f7fafc232d42-kube-api-access-ssk24\" (UniqueName: \"kubernetes.io/projected/5b44b714-aa08-4392-b4f4-f7fafc232d42-kube-api-access-ssk24\") pod \"5b44b714-aa08-4392-b4f4-f7fafc232d42\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") "
Nov 04 02:21:56.282262 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.282139    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b44b714-aa08-4392-b4f4-f7fafc232d42-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b44b714-aa08-4392-b4f4-f7fafc232d42-tls-certs\") pod \"5b44b714-aa08-4392-b4f4-f7fafc232d42\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") "
Nov 04 02:21:56.282325 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.282280    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-home\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-home\") pod \"5b44b714-aa08-4392-b4f4-f7fafc232d42\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") "
Nov 04 02:21:56.282379 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.282350    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-model-cache\") pod \"5b44b714-aa08-4392-b4f4-f7fafc232d42\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") "
Nov 04 02:21:56.282441 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.282423    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-dshm\") pod \"5b44b714-aa08-4392-b4f4-f7fafc232d42\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") "
Nov 04 02:21:56.282520 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.282459    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-tmp-dir\") pod \"5b44b714-aa08-4392-b4f4-f7fafc232d42\" (UID: \"5b44b714-aa08-4392-b4f4-f7fafc232d42\") "
Nov 04 02:21:56.282578 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.282548    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-home" pod "5b44b714-aa08-4392-b4f4-f7fafc232d42" (UID: "5b44b714-aa08-4392-b4f4-f7fafc232d42"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.282632 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.282606    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-model-cache" pod "5b44b714-aa08-4392-b4f4-f7fafc232d42" (UID: "5b44b714-aa08-4392-b4f4-f7fafc232d42"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.282846 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.282827    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-tmp-dir" pod "5b44b714-aa08-4392-b4f4-f7fafc232d42" (UID: "5b44b714-aa08-4392-b4f4-f7fafc232d42"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.283126 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.283105    4512 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-home\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.283190 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.283134    4512 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-model-cache\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.283190 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.283177    4512 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-tmp-dir\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.284614 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.284590    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b44b714-aa08-4392-b4f4-f7fafc232d42-tls-certs" pod "5b44b714-aa08-4392-b4f4-f7fafc232d42" (UID: "5b44b714-aa08-4392-b4f4-f7fafc232d42"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:56.285050 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.285029    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-dshm" pod "5b44b714-aa08-4392-b4f4-f7fafc232d42" (UID: "5b44b714-aa08-4392-b4f4-f7fafc232d42"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:56.285122 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.285069    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b44b714-aa08-4392-b4f4-f7fafc232d42-kube-api-access-ssk24" pod "5b44b714-aa08-4392-b4f4-f7fafc232d42" (UID: "5b44b714-aa08-4392-b4f4-f7fafc232d42"). InnerVolumeSpecName "kube-api-access-ssk24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:56.388814 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.385207    4512 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b44b714-aa08-4392-b4f4-f7fafc232d42-dshm\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.388814 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.385245    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssk24\" (UniqueName: \"kubernetes.io/projected/5b44b714-aa08-4392-b4f4-f7fafc232d42-kube-api-access-ssk24\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.388814 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.385260    4512 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b44b714-aa08-4392-b4f4-f7fafc232d42-tls-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.633702 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.633660    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:56.698218 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.698178    4512 generic.go:356] "Generic (PLEG): container finished" podID="5b44b714-aa08-4392-b4f4-f7fafc232d42" containerID="f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252" exitCode=0
Nov 04 02:21:56.698711 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.698230    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"
Nov 04 02:21:56.698711 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.698253    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j" event={"ID":"5b44b714-aa08-4392-b4f4-f7fafc232d42","Type":"ContainerDied","Data":"f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252"}
Nov 04 02:21:56.698711 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.698301    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j" event={"ID":"5b44b714-aa08-4392-b4f4-f7fafc232d42","Type":"ContainerDied","Data":"34305e8041931754401ce4e472d1fa1f283cca556a30b96c7ab1a43b01659edc"}
Nov 04 02:21:56.698711 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.698320    4512 scope.go:122] "RemoveContainer" containerID="f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252"
Nov 04 02:21:56.700968 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.700942    4512 generic.go:356] "Generic (PLEG): container finished" podID="872308d5-8d98-49c7-8424-eddc7de82b41" containerID="ff10794673a17e788e7a32ea9302a28775f5fb9bc9247dfeb6e757f1bfcf5789" exitCode=0
Nov 04 02:21:56.701259 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.701193    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv" event={"ID":"872308d5-8d98-49c7-8424-eddc7de82b41","Type":"ContainerDied","Data":"ff10794673a17e788e7a32ea9302a28775f5fb9bc9247dfeb6e757f1bfcf5789"}
Nov 04 02:21:56.711179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.711155    4512 scope.go:122] "RemoveContainer" containerID="f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252"
Nov 04 02:21:56.711649 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:21:56.711606    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252\": container with ID starting with f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252 not found: ID does not exist" containerID="f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252"
Nov 04 02:21:56.711769 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.711649    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252"} err="failed to get container status \"f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252\": rpc error: code = NotFound desc = could not find container \"f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252\": container with ID starting with f54aa0f2f779ea9db0be7d3e851293c1befbd4577b23236661c702f4cafa5252 not found: ID does not exist"
Nov 04 02:21:56.788836 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:21:56.788767    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:21:56.863377 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.863351    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:56.979296 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.979207    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"]
Nov 04 02:21:56.983072 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.983047    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/distinct-2-w0-24bea9-kserve-f86b4f847-g5w2j"]
Nov 04 02:21:56.991890 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.991857    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-xks-ca\") pod \"689d222a-ebe1-4493-b3ef-08c76a8678b3\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") "
Nov 04 02:21:56.991978 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.991966    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-openshift-ca\") pod \"689d222a-ebe1-4493-b3ef-08c76a8678b3\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") "
Nov 04 02:21:56.992019 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.992007    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/689d222a-ebe1-4493-b3ef-08c76a8678b3-kube-api-access-jrljg\" (UniqueName: \"kubernetes.io/projected/689d222a-ebe1-4493-b3ef-08c76a8678b3-kube-api-access-jrljg\") pod \"689d222a-ebe1-4493-b3ef-08c76a8678b3\" (UID: \"689d222a-ebe1-4493-b3ef-08c76a8678b3\") "
Nov 04 02:21:56.992349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.992320    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-openshift-ca" pod "689d222a-ebe1-4493-b3ef-08c76a8678b3" (UID: "689d222a-ebe1-4493-b3ef-08c76a8678b3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:56.992449 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.992435    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:56.994236 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.994200    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-xks-ca" pod "689d222a-ebe1-4493-b3ef-08c76a8678b3" (UID: "689d222a-ebe1-4493-b3ef-08c76a8678b3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:56.994386 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:56.994362    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/689d222a-ebe1-4493-b3ef-08c76a8678b3-kube-api-access-jrljg" pod "689d222a-ebe1-4493-b3ef-08c76a8678b3" (UID: "689d222a-ebe1-4493-b3ef-08c76a8678b3"). InnerVolumeSpecName "kube-api-access-jrljg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:57.093577 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:57.093529    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/689d222a-ebe1-4493-b3ef-08c76a8678b3-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:57.093577 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:57.093581    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrljg\" (UniqueName: \"kubernetes.io/projected/689d222a-ebe1-4493-b3ef-08c76a8678b3-kube-api-access-jrljg\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:57.709803 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:57.709776    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"
Nov 04 02:21:57.709803 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:57.709789    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj" event={"ID":"689d222a-ebe1-4493-b3ef-08c76a8678b3","Type":"ContainerDied","Data":"f1673de711a4a96b6e119663d0aed139521ce63e43115149d2d7504356a2a742"}
Nov 04 02:21:57.710349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:57.709853    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1673de711a4a96b6e119663d0aed139521ce63e43115149d2d7504356a2a742"
Nov 04 02:21:57.861113 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:57.861087    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:57.980327 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:57.980235    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg" podUID="485907b4-566a-48a7-a669-d3a293bb0d29" containerName="payload-processing" containerID="cri-o://368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3" gracePeriod=25
Nov 04 02:21:58.005722 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.005689    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/872308d5-8d98-49c7-8424-eddc7de82b41-kube-api-access-2djjt\" (UniqueName: \"kubernetes.io/projected/872308d5-8d98-49c7-8424-eddc7de82b41-kube-api-access-2djjt\") pod \"872308d5-8d98-49c7-8424-eddc7de82b41\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") "
Nov 04 02:21:58.005929 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.005750    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-openshift-ca\") pod \"872308d5-8d98-49c7-8424-eddc7de82b41\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") "
Nov 04 02:21:58.005929 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.005889    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-xks-ca\") pod \"872308d5-8d98-49c7-8424-eddc7de82b41\" (UID: \"872308d5-8d98-49c7-8424-eddc7de82b41\") "
Nov 04 02:21:58.006309 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.006185    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-openshift-ca" pod "872308d5-8d98-49c7-8424-eddc7de82b41" (UID: "872308d5-8d98-49c7-8424-eddc7de82b41"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:58.006465 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.006441    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:58.008349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.008322    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872308d5-8d98-49c7-8424-eddc7de82b41-kube-api-access-2djjt" pod "872308d5-8d98-49c7-8424-eddc7de82b41" (UID: "872308d5-8d98-49c7-8424-eddc7de82b41"). InnerVolumeSpecName "kube-api-access-2djjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:58.008630 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.008603    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-xks-ca" pod "872308d5-8d98-49c7-8424-eddc7de82b41" (UID: "872308d5-8d98-49c7-8424-eddc7de82b41"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:58.087007 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.086963    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh" podUID="989b004f-6a28-4e46-9bd4-97e6d27df64f" containerName="payload-pre-processing" containerID="cri-o://56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1" gracePeriod=25
Nov 04 02:21:58.107480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.107447    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2djjt\" (UniqueName: \"kubernetes.io/projected/872308d5-8d98-49c7-8424-eddc7de82b41-kube-api-access-2djjt\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:58.107480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.107479    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/872308d5-8d98-49c7-8424-eddc7de82b41-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:58.289281 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.289229    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:58.314411 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.314368    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b44b714-aa08-4392-b4f4-f7fafc232d42" path="/var/lib/kubelet/pods/5b44b714-aa08-4392-b4f4-f7fafc232d42/volumes"
Nov 04 02:21:58.720956 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.720862    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv" event={"ID":"872308d5-8d98-49c7-8424-eddc7de82b41","Type":"ContainerDied","Data":"f9e947382e0077ea03e816b3ec6f599dc1a74068f2fa6b5f14cae5064a0c26e7"}
Nov 04 02:21:58.720956 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.720896    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"
Nov 04 02:21:58.720956 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:58.720923    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9e947382e0077ea03e816b3ec6f599dc1a74068f2fa6b5f14cae5064a0c26e7"
Nov 04 02:21:59.544006 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:21:59.543968    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:01.585349 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:01.585199    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:05.747142 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:05.747092    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:06.153970 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:06.153870    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg" podUID="485907b4-566a-48a7-a669-d3a293bb0d29" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:06.154161 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:06.153892    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh" podUID="989b004f-6a28-4e46-9bd4-97e6d27df64f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:06.677193 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:06.677156    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:08.995697 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:08.995661    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:09.221963 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:09.221926    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:10.841659 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:10.841623    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:11.091089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.091051    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:11.390603 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.390569    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"]
Nov 04 02:22:11.391529 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.391495    4512 cpu_manager.go:414] "containerMap: removing container" podUID="872308d5-8d98-49c7-8424-eddc7de82b41" containerName="revoke-keys"
Nov 04 02:22:11.391592 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.391533    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="872308d5-8d98-49c7-8424-eddc7de82b41" containerName="revoke-keys"
Nov 04 02:22:11.391592 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.391568    4512 cpu_manager.go:414] "containerMap: removing container" podUID="689d222a-ebe1-4493-b3ef-08c76a8678b3" containerName="revoke-keys"
Nov 04 02:22:11.391592 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.391576    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="689d222a-ebe1-4493-b3ef-08c76a8678b3" containerName="revoke-keys"
Nov 04 02:22:11.391689 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.391616    4512 cpu_manager.go:414] "containerMap: removing container" podUID="5b44b714-aa08-4392-b4f4-f7fafc232d42" containerName="main"
Nov 04 02:22:11.391689 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.391624    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b44b714-aa08-4392-b4f4-f7fafc232d42" containerName="main"
Nov 04 02:22:11.391758 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.391746    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="872308d5-8d98-49c7-8424-eddc7de82b41" containerName="revoke-keys"
Nov 04 02:22:11.391790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.391764    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="689d222a-ebe1-4493-b3ef-08c76a8678b3" containerName="revoke-keys"
Nov 04 02:22:11.391790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.391781    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b44b714-aa08-4392-b4f4-f7fafc232d42" containerName="main"
Nov 04 02:22:11.397129 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.397100    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:11.449142 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.449109    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:22:11.458668 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.458634    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:22:11.469460 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.469426    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a2405010-lpnfs\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:11.469643 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.469589    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4tlc\" (UniqueName: \"kubernetes.io/projected/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-kube-api-access-d4tlc\") pod \"maas-api-revoke-sub-a2405010-lpnfs\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:11.469691 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.469669    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a2405010-lpnfs\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:11.502893 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.502854    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"]
Nov 04 02:22:11.571081 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.571042    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4tlc\" (UniqueName: \"kubernetes.io/projected/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-kube-api-access-d4tlc\") pod \"maas-api-revoke-sub-a2405010-lpnfs\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:11.571081 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.571081    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a2405010-lpnfs\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:11.571288 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.571178    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a2405010-lpnfs\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:11.571832 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.571806    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a2405010-lpnfs\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:11.573701 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.573682    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a2405010-lpnfs\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:11.632255 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.632218    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4tlc\" (UniqueName: \"kubernetes.io/projected/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-kube-api-access-d4tlc\") pod \"maas-api-revoke-sub-a2405010-lpnfs\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:11.712715 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:11.712633    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:12.057926 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:12.057891    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"]
Nov 04 02:22:12.061466 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:12.061444    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9c237ef8-tgn2t"]
Nov 04 02:22:12.106445 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:12.106412    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"]
Nov 04 02:22:12.136939 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:22:12.136908    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7edbc9_f82b_404e_9eaf_e5bba4a00f8a.slice/crio-b21fabbae1178d6187b39a92e53b3ad4d64f5ed41d5e1d471733941097fc924a WatchSource:0}: Error finding container b21fabbae1178d6187b39a92e53b3ad4d64f5ed41d5e1d471733941097fc924a: Status 404 returned error can't find the container with id b21fabbae1178d6187b39a92e53b3ad4d64f5ed41d5e1d471733941097fc924a
Nov 04 02:22:12.313286 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:12.313205    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26be670d-5b5e-4a6b-b623-ac721ceb4fda" path="/var/lib/kubelet/pods/26be670d-5b5e-4a6b-b623-ac721ceb4fda/volumes"
Nov 04 02:22:12.768328 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:12.768291    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:12.882086 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:12.882047    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs" event={"ID":"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a","Type":"ContainerStarted","Data":"749654ae58f210ddd66a7affba6990bfd14134289913d3ced4cc341be43b90fc"}
Nov 04 02:22:12.882274 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:12.882103    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs" event={"ID":"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a","Type":"ContainerStarted","Data":"b21fabbae1178d6187b39a92e53b3ad4d64f5ed41d5e1d471733941097fc924a"}
Nov 04 02:22:12.999932 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:12.999875    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs" podStartSLOduration=1.999861822 podStartE2EDuration="1.999861822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:22:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:22:12.996644743 +0000 UTC m=+1594.802101376" watchObservedRunningTime="2026-11-04 02:22:12.999861822 +0000 UTC m=+1594.805318454"
Nov 04 02:22:13.890994 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:13.890951    4512 generic.go:356] "Generic (PLEG): container finished" podID="ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a" containerID="749654ae58f210ddd66a7affba6990bfd14134289913d3ced4cc341be43b90fc" exitCode=0
Nov 04 02:22:13.891383 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:13.891043    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs" event={"ID":"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a","Type":"ContainerDied","Data":"749654ae58f210ddd66a7affba6990bfd14134289913d3ced4cc341be43b90fc"}
Nov 04 02:22:15.048199 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.048176    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:15.110899 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.110867    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-kube-api-access-d4tlc\" (UniqueName: \"kubernetes.io/projected/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-kube-api-access-d4tlc\") pod \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") "
Nov 04 02:22:15.111053 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.110955    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-xks-ca\") pod \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") "
Nov 04 02:22:15.111053 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.111002    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-openshift-ca\") pod \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\" (UID: \"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a\") "
Nov 04 02:22:15.111450 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.111421    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-openshift-ca" pod "ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a" (UID: "ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:15.111743 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.111690    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:15.113284 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.113257    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-xks-ca" pod "ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a" (UID: "ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:15.113415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.113390    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-kube-api-access-d4tlc" pod "ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a" (UID: "ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a"). InnerVolumeSpecName "kube-api-access-d4tlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:15.212787 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.212705    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4tlc\" (UniqueName: \"kubernetes.io/projected/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-kube-api-access-d4tlc\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:15.212787 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.212738    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:15.743886 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.743850    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"]
Nov 04 02:22:15.793871 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.793825    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"]
Nov 04 02:22:15.910404 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.910376    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"
Nov 04 02:22:15.910404 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.910390    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs" event={"ID":"ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a","Type":"ContainerDied","Data":"b21fabbae1178d6187b39a92e53b3ad4d64f5ed41d5e1d471733941097fc924a"}
Nov 04 02:22:15.910702 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.910433    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b21fabbae1178d6187b39a92e53b3ad4d64f5ed41d5e1d471733941097fc924a"
Nov 04 02:22:15.960759 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:15.960720    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:16.153762 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:16.153685    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg" podUID="485907b4-566a-48a7-a669-d3a293bb0d29" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:16.154122 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:16.153681    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh" podUID="989b004f-6a28-4e46-9bd4-97e6d27df64f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:17.591656 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:17.591615    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:18.073901 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.073867    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:18.281409 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.281243    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:18.302131 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.302102    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:22:18.350371 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.350347    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/485907b4-566a-48a7-a669-d3a293bb0d29-kube-api-access-q7ks8\" (UniqueName: \"kubernetes.io/projected/485907b4-566a-48a7-a669-d3a293bb0d29-kube-api-access-q7ks8\") pod \"485907b4-566a-48a7-a669-d3a293bb0d29\" (UID: \"485907b4-566a-48a7-a669-d3a293bb0d29\") "
Nov 04 02:22:18.350542 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.350491    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/485907b4-566a-48a7-a669-d3a293bb0d29-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/485907b4-566a-48a7-a669-d3a293bb0d29-plugins-config-volume\") pod \"485907b4-566a-48a7-a669-d3a293bb0d29\" (UID: \"485907b4-566a-48a7-a669-d3a293bb0d29\") "
Nov 04 02:22:18.350875 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.350854    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/485907b4-566a-48a7-a669-d3a293bb0d29-plugins-config-volume" pod "485907b4-566a-48a7-a669-d3a293bb0d29" (UID: "485907b4-566a-48a7-a669-d3a293bb0d29"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:18.351004 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.350989    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/485907b4-566a-48a7-a669-d3a293bb0d29-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:18.352781 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.352759    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/485907b4-566a-48a7-a669-d3a293bb0d29-kube-api-access-q7ks8" pod "485907b4-566a-48a7-a669-d3a293bb0d29" (UID: "485907b4-566a-48a7-a669-d3a293bb0d29"). InnerVolumeSpecName "kube-api-access-q7ks8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:18.360989 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.360971    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:22:18.451324 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.451292    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/989b004f-6a28-4e46-9bd4-97e6d27df64f-kube-api-access-wr7zv\" (UniqueName: \"kubernetes.io/projected/989b004f-6a28-4e46-9bd4-97e6d27df64f-kube-api-access-wr7zv\") pod \"989b004f-6a28-4e46-9bd4-97e6d27df64f\" (UID: \"989b004f-6a28-4e46-9bd4-97e6d27df64f\") "
Nov 04 02:22:18.451549 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.451399    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/989b004f-6a28-4e46-9bd4-97e6d27df64f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/989b004f-6a28-4e46-9bd4-97e6d27df64f-plugins-config-volume\") pod \"989b004f-6a28-4e46-9bd4-97e6d27df64f\" (UID: \"989b004f-6a28-4e46-9bd4-97e6d27df64f\") "
Nov 04 02:22:18.451837 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.451809    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/989b004f-6a28-4e46-9bd4-97e6d27df64f-plugins-config-volume" pod "989b004f-6a28-4e46-9bd4-97e6d27df64f" (UID: "989b004f-6a28-4e46-9bd4-97e6d27df64f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:18.451947 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.451931    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7ks8\" (UniqueName: \"kubernetes.io/projected/485907b4-566a-48a7-a669-d3a293bb0d29-kube-api-access-q7ks8\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:18.451947 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.451945    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/989b004f-6a28-4e46-9bd4-97e6d27df64f-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:18.454245 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.454217    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/989b004f-6a28-4e46-9bd4-97e6d27df64f-kube-api-access-wr7zv" pod "989b004f-6a28-4e46-9bd4-97e6d27df64f" (UID: "989b004f-6a28-4e46-9bd4-97e6d27df64f"). InnerVolumeSpecName "kube-api-access-wr7zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:18.553123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.553038    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr7zv\" (UniqueName: \"kubernetes.io/projected/989b004f-6a28-4e46-9bd4-97e6d27df64f-kube-api-access-wr7zv\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:18.670728 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.670694    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:18.936019 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.935932    4512 generic.go:356] "Generic (PLEG): container finished" podID="485907b4-566a-48a7-a669-d3a293bb0d29" containerID="368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3" exitCode=0
Nov 04 02:22:18.936019 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.935989    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"
Nov 04 02:22:18.936223 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.936023    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg" event={"ID":"485907b4-566a-48a7-a669-d3a293bb0d29","Type":"ContainerDied","Data":"368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3"}
Nov 04 02:22:18.936223 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.936112    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg" event={"ID":"485907b4-566a-48a7-a669-d3a293bb0d29","Type":"ContainerDied","Data":"e64016a564a915264e4755d6390c8b127d6c18c1d568b449c63384773a9a6ebe"}
Nov 04 02:22:18.936223 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.936139    4512 scope.go:122] "RemoveContainer" containerID="368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3"
Nov 04 02:22:18.937483 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.937458    4512 generic.go:356] "Generic (PLEG): container finished" podID="989b004f-6a28-4e46-9bd4-97e6d27df64f" containerID="56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1" exitCode=0
Nov 04 02:22:18.937636 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.937544    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"
Nov 04 02:22:18.937636 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.937548    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh" event={"ID":"989b004f-6a28-4e46-9bd4-97e6d27df64f","Type":"ContainerDied","Data":"56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1"}
Nov 04 02:22:18.937636 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.937609    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh" event={"ID":"989b004f-6a28-4e46-9bd4-97e6d27df64f","Type":"ContainerDied","Data":"c9bcb6a291fa0c8054db8b2898c11639d2080a024d0257b1074acd7d7088d94a"}
Nov 04 02:22:18.948181 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.948166    4512 scope.go:122] "RemoveContainer" containerID="368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3"
Nov 04 02:22:18.948425 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:22:18.948407    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3\": container with ID starting with 368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3 not found: ID does not exist" containerID="368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3"
Nov 04 02:22:18.948493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.948435    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3"} err="failed to get container status \"368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3\": rpc error: code = NotFound desc = could not find container \"368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3\": container with ID starting with 368551b72d22f75a7331c2819ff8822aec2dc693c90b6baedb4cd0b660e78bc3 not found: ID does not exist"
Nov 04 02:22:18.948493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.948455    4512 scope.go:122] "RemoveContainer" containerID="56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1"
Nov 04 02:22:18.962547 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.962498    4512 scope.go:122] "RemoveContainer" containerID="56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1"
Nov 04 02:22:18.962918 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:22:18.962893    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1\": container with ID starting with 56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1 not found: ID does not exist" containerID="56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1"
Nov 04 02:22:18.963028 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:18.962941    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1"} err="failed to get container status \"56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1\": rpc error: code = NotFound desc = could not find container \"56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1\": container with ID starting with 56036cd952dd74b5005581f47ad029f48d306bd55089b9f3c3fc447296dda0e1 not found: ID does not exist"
Nov 04 02:22:19.102193 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:19.102154    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"]
Nov 04 02:22:19.151647 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:19.151605    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg"]
Nov 04 02:22:19.214862 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:19.214782    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"]
Nov 04 02:22:19.218344 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:19.218322    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh"]
Nov 04 02:22:20.313049 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:20.313015    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="485907b4-566a-48a7-a669-d3a293bb0d29" path="/var/lib/kubelet/pods/485907b4-566a-48a7-a669-d3a293bb0d29/volumes"
Nov 04 02:22:20.313496 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:20.313478    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="989b004f-6a28-4e46-9bd4-97e6d27df64f" path="/var/lib/kubelet/pods/989b004f-6a28-4e46-9bd4-97e6d27df64f/volumes"
Nov 04 02:22:20.705318 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:20.705226    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:20.924925 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:20.924874    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:22.059762 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:22.059715    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:22.257348 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:22.257296    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:22.687983 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:22.687948    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"]
Nov 04 02:22:22.688435 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:22.688399    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy" containerID="cri-o://5008dab405e9209988b7d6b5f2d1f65c342e15858b5e0e01e25aa12e84cbd437" gracePeriod=30
Nov 04 02:22:22.688633 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:22.688577    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:22.940842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:22.940754    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:24.704141 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:24.704096    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:26.871400 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:22:26.871354    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:22:27.062407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.061954    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"]
Nov 04 02:22:27.178404 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.178304    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-72bf9b8f-c6m9h"]
Nov 04 02:22:27.243582 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.237339    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"]
Nov 04 02:22:27.244746 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.244712    4512 cpu_manager.go:414] "containerMap: removing container" podUID="ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a" containerName="revoke-keys"
Nov 04 02:22:27.244913 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.244898    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a" containerName="revoke-keys"
Nov 04 02:22:27.245065 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.245054    4512 cpu_manager.go:414] "containerMap: removing container" podUID="989b004f-6a28-4e46-9bd4-97e6d27df64f" containerName="payload-pre-processing"
Nov 04 02:22:27.245151 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.245140    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="989b004f-6a28-4e46-9bd4-97e6d27df64f" containerName="payload-pre-processing"
Nov 04 02:22:27.245229 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.245218    4512 cpu_manager.go:414] "containerMap: removing container" podUID="485907b4-566a-48a7-a669-d3a293bb0d29" containerName="payload-processing"
Nov 04 02:22:27.245312 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.245299    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="485907b4-566a-48a7-a669-d3a293bb0d29" containerName="payload-processing"
Nov 04 02:22:27.245612 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.245593    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="485907b4-566a-48a7-a669-d3a293bb0d29" containerName="payload-processing"
Nov 04 02:22:27.245747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.245734    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a" containerName="revoke-keys"
Nov 04 02:22:27.245840 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.245830    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="989b004f-6a28-4e46-9bd4-97e6d27df64f" containerName="payload-pre-processing"
Nov 04 02:22:27.274463 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.271967    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.280449 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.280412    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"]
Nov 04 02:22:27.334901 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.334855    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:22:27.373537 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.336110    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:22:27.384124 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.384080    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lgbk\" (UniqueName: \"kubernetes.io/projected/d31b6b0c-f769-4a79-a11f-5689b9deba4d-kube-api-access-9lgbk\") pod \"maas-api-revoke-sub-b3c0faf9-9hjxf\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.384441 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.384416    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b3c0faf9-9hjxf\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.384680 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.384661    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b3c0faf9-9hjxf\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.485813 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.485762    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lgbk\" (UniqueName: \"kubernetes.io/projected/d31b6b0c-f769-4a79-a11f-5689b9deba4d-kube-api-access-9lgbk\") pod \"maas-api-revoke-sub-b3c0faf9-9hjxf\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.486006 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.485889    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b3c0faf9-9hjxf\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.486084 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.486006    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b3c0faf9-9hjxf\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.487249 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.487211    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b3c0faf9-9hjxf\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.543528 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.537964    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b3c0faf9-9hjxf\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.577678 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.577639    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lgbk\" (UniqueName: \"kubernetes.io/projected/d31b6b0c-f769-4a79-a11f-5689b9deba4d-kube-api-access-9lgbk\") pod \"maas-api-revoke-sub-b3c0faf9-9hjxf\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.623743 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.623703    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:27.671644 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.671582    4512 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:22:27.671780 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:27.671712    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:22:28.063597 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.058600    4512 generic.go:356] "Generic (PLEG): container finished" podID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerID="5008dab405e9209988b7d6b5f2d1f65c342e15858b5e0e01e25aa12e84cbd437" exitCode=0
Nov 04 02:22:28.063597 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.058824    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" event={"ID":"5c7e0fd5-eece-41b5-94e0-a25a09f28332","Type":"ContainerDied","Data":"5008dab405e9209988b7d6b5f2d1f65c342e15858b5e0e01e25aa12e84cbd437"}
Nov 04 02:22:28.220881 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.220698    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:28.272282 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.272248    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:22:28.315109 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.314996    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d06e156c-8dcc-4d4b-b731-24b4615291d9" path="/var/lib/kubelet/pods/d06e156c-8dcc-4d4b-b731-24b4615291d9/volumes"
Nov 04 02:22:28.318113 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318088    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-kube-api-access-6m8rg\" (UniqueName: \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-kube-api-access-6m8rg\") pod \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") "
Nov 04 02:22:28.318261 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318129    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-envoy\") pod \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") "
Nov 04 02:22:28.318261 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318163    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istiod-ca-cert\") pod \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") "
Nov 04 02:22:28.318261 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318191    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-data\") pod \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") "
Nov 04 02:22:28.318415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318270    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-credential-socket\") pod \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") "
Nov 04 02:22:28.318415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318299    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-token\" (UniqueName: \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-token\") pod \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") "
Nov 04 02:22:28.318415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318330    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-certs\") pod \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") "
Nov 04 02:22:28.318605 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318460    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-podinfo\") pod \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") "
Nov 04 02:22:28.318605 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318549    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-socket\") pod \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\" (UID: \"5c7e0fd5-eece-41b5-94e0-a25a09f28332\") "
Nov 04 02:22:28.318605 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318541    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-credential-socket" pod "5c7e0fd5-eece-41b5-94e0-a25a09f28332" (UID: "5c7e0fd5-eece-41b5-94e0-a25a09f28332"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:28.318762 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318599    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-data" pod "5c7e0fd5-eece-41b5-94e0-a25a09f28332" (UID: "5c7e0fd5-eece-41b5-94e0-a25a09f28332"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:28.318762 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318617    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istiod-ca-cert" pod "5c7e0fd5-eece-41b5-94e0-a25a09f28332" (UID: "5c7e0fd5-eece-41b5-94e0-a25a09f28332"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:28.318762 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318742    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-certs" pod "5c7e0fd5-eece-41b5-94e0-a25a09f28332" (UID: "5c7e0fd5-eece-41b5-94e0-a25a09f28332"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:28.318923 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318906    4512 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istiod-ca-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.318991 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318950    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-socket" pod "5c7e0fd5-eece-41b5-94e0-a25a09f28332" (UID: "5c7e0fd5-eece-41b5-94e0-a25a09f28332"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:28.318991 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.318945    4512 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-data\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.319124 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.319068    4512 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-credential-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.319235 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.319126    4512 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.320390 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.320365    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-envoy" pod "5c7e0fd5-eece-41b5-94e0-a25a09f28332" (UID: "5c7e0fd5-eece-41b5-94e0-a25a09f28332"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:28.320595 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.320572    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-token" pod "5c7e0fd5-eece-41b5-94e0-a25a09f28332" (UID: "5c7e0fd5-eece-41b5-94e0-a25a09f28332"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:28.320665 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.320647    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-kube-api-access-6m8rg" pod "5c7e0fd5-eece-41b5-94e0-a25a09f28332" (UID: "5c7e0fd5-eece-41b5-94e0-a25a09f28332"). InnerVolumeSpecName "kube-api-access-6m8rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:28.320730 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.320704    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-podinfo" pod "5c7e0fd5-eece-41b5-94e0-a25a09f28332" (UID: "5c7e0fd5-eece-41b5-94e0-a25a09f28332"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:22:28.379192 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.379162    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"]
Nov 04 02:22:28.380271 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:22:28.380243    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31b6b0c_f769_4a79_a11f_5689b9deba4d.slice/crio-16da126c002e9c3a00b076069723c6890632b0fa17a5f68c82cc19df43fc2349 WatchSource:0}: Error finding container 16da126c002e9c3a00b076069723c6890632b0fa17a5f68c82cc19df43fc2349: Status 404 returned error can't find the container with id 16da126c002e9c3a00b076069723c6890632b0fa17a5f68c82cc19df43fc2349
Nov 04 02:22:28.420279 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.420247    4512 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-podinfo\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.420279 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.420279    4512 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-workload-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.420521 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.420292    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m8rg\" (UniqueName: \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-kube-api-access-6m8rg\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.420521 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.420303    4512 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-envoy\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.420521 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.420318    4512 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c7e0fd5-eece-41b5-94e0-a25a09f28332-istio-token\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:28.782881 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.782837    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:28.907407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:28.907318    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"]
Nov 04 02:22:29.018350 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.018237    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"]
Nov 04 02:22:29.068738 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.068698    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf" event={"ID":"d31b6b0c-f769-4a79-a11f-5689b9deba4d","Type":"ContainerStarted","Data":"af115a15f78e35a5a2aecc42b291e4d61ae5b9d6a338131767e072f6c3c70720"}
Nov 04 02:22:29.069226 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.068763    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf" event={"ID":"d31b6b0c-f769-4a79-a11f-5689b9deba4d","Type":"ContainerStarted","Data":"16da126c002e9c3a00b076069723c6890632b0fa17a5f68c82cc19df43fc2349"}
Nov 04 02:22:29.070336 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.070309    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd" event={"ID":"5c7e0fd5-eece-41b5-94e0-a25a09f28332","Type":"ContainerDied","Data":"9e09453544fa8b5439389837fb580821ee00219d4b30c2d0d6e63f109fb5b5ae"}
Nov 04 02:22:29.070336 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.070327    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"
Nov 04 02:22:29.070528 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.070355    4512 scope.go:122] "RemoveContainer" containerID="5008dab405e9209988b7d6b5f2d1f65c342e15858b5e0e01e25aa12e84cbd437"
Nov 04 02:22:29.256825 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.256773    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf" podStartSLOduration=2.256758355 podStartE2EDuration="2.256758355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:22:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:22:29.252705133 +0000 UTC m=+1611.058161791" watchObservedRunningTime="2026-11-04 02:22:29.256758355 +0000 UTC m=+1611.062214987"
Nov 04 02:22:29.337056 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.337023    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:29.372607 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.372559    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"]
Nov 04 02:22:29.377186 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.377160    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd"]
Nov 04 02:22:29.538978 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:29.538900    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:30.079566 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:30.079529    4512 generic.go:356] "Generic (PLEG): container finished" podID="d31b6b0c-f769-4a79-a11f-5689b9deba4d" containerID="af115a15f78e35a5a2aecc42b291e4d61ae5b9d6a338131767e072f6c3c70720" exitCode=0
Nov 04 02:22:30.079975 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:30.079616    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf" event={"ID":"d31b6b0c-f769-4a79-a11f-5689b9deba4d","Type":"ContainerDied","Data":"af115a15f78e35a5a2aecc42b291e4d61ae5b9d6a338131767e072f6c3c70720"}
Nov 04 02:22:30.314725 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:30.314686    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" path="/var/lib/kubelet/pods/5c7e0fd5-eece-41b5-94e0-a25a09f28332/volumes"
Nov 04 02:22:30.766832 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:30.766779    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs" podUID="31069c58-7113-4e77-9672-9862fd6576da" containerName="main" containerID="cri-o://a48a838e35ec1bb5e612db79ddc8ed7c5545bb7e786b4879c3cc45b84fdd88f7" gracePeriod=45
Nov 04 02:22:30.816335 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:30.816287    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r" podUID="d93d5ffe-05d3-444c-a816-87c0822c0ce4" containerName="main" containerID="cri-o://681cb37b7f865e9f309db1f8127413fd039c4ce36e7663cf1acdb2af832c6bbf" gracePeriod=45
Nov 04 02:22:30.836452 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:30.836423    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:31.095115 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.094626    4512 generic.go:356] "Generic (PLEG): container finished" podID="31069c58-7113-4e77-9672-9862fd6576da" containerID="a48a838e35ec1bb5e612db79ddc8ed7c5545bb7e786b4879c3cc45b84fdd88f7" exitCode=0
Nov 04 02:22:31.095115 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.094756    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs" event={"ID":"31069c58-7113-4e77-9672-9862fd6576da","Type":"ContainerDied","Data":"a48a838e35ec1bb5e612db79ddc8ed7c5545bb7e786b4879c3cc45b84fdd88f7"}
Nov 04 02:22:31.097738 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.097330    4512 generic.go:356] "Generic (PLEG): container finished" podID="d93d5ffe-05d3-444c-a816-87c0822c0ce4" containerID="681cb37b7f865e9f309db1f8127413fd039c4ce36e7663cf1acdb2af832c6bbf" exitCode=0
Nov 04 02:22:31.097738 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.097684    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r" event={"ID":"d93d5ffe-05d3-444c-a816-87c0822c0ce4","Type":"ContainerDied","Data":"681cb37b7f865e9f309db1f8127413fd039c4ce36e7663cf1acdb2af832c6bbf"}
Nov 04 02:22:31.125253 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.125095    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:31.176941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.176001    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:22:31.224047 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.223614    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259145    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d93d5ffe-05d3-444c-a816-87c0822c0ce4-kube-api-access-qwjkd\" (UniqueName: \"kubernetes.io/projected/d93d5ffe-05d3-444c-a816-87c0822c0ce4-kube-api-access-qwjkd\") pod \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259220    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-model-cache\") pod \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259257    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-dshm\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-dshm\") pod \"31069c58-7113-4e77-9672-9862fd6576da\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259302    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/31069c58-7113-4e77-9672-9862fd6576da-tls-certs\" (UniqueName: \"kubernetes.io/secret/31069c58-7113-4e77-9672-9862fd6576da-tls-certs\") pod \"31069c58-7113-4e77-9672-9862fd6576da\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259378    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-home\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-home\") pod \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259418    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-model-cache\") pod \"31069c58-7113-4e77-9672-9862fd6576da\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259459    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-home\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-home\") pod \"31069c58-7113-4e77-9672-9862fd6576da\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259496    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tmp-dir\") pod \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259555    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tls-certs\" (UniqueName: \"kubernetes.io/secret/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tls-certs\") pod \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259616    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31069c58-7113-4e77-9672-9862fd6576da-kube-api-access-pp94q\" (UniqueName: \"kubernetes.io/projected/31069c58-7113-4e77-9672-9862fd6576da-kube-api-access-pp94q\") pod \"31069c58-7113-4e77-9672-9862fd6576da\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259660    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-dshm\") pod \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\" (UID: \"d93d5ffe-05d3-444c-a816-87c0822c0ce4\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.259808    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-tmp-dir\") pod \"31069c58-7113-4e77-9672-9862fd6576da\" (UID: \"31069c58-7113-4e77-9672-9862fd6576da\") "
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.260742    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-tmp-dir" pod "31069c58-7113-4e77-9672-9862fd6576da" (UID: "31069c58-7113-4e77-9672-9862fd6576da"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.266077    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-model-cache" pod "d93d5ffe-05d3-444c-a816-87c0822c0ce4" (UID: "d93d5ffe-05d3-444c-a816-87c0822c0ce4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.266964    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tmp-dir" pod "d93d5ffe-05d3-444c-a816-87c0822c0ce4" (UID: "d93d5ffe-05d3-444c-a816-87c0822c0ce4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.267608 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.267577    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-model-cache" pod "31069c58-7113-4e77-9672-9862fd6576da" (UID: "31069c58-7113-4e77-9672-9862fd6576da"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.268437 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.267753    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-home" pod "31069c58-7113-4e77-9672-9862fd6576da" (UID: "31069c58-7113-4e77-9672-9862fd6576da"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.268581 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.268558    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-home" pod "d93d5ffe-05d3-444c-a816-87c0822c0ce4" (UID: "d93d5ffe-05d3-444c-a816-87c0822c0ce4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.268696 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.268671    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-dshm" pod "31069c58-7113-4e77-9672-9862fd6576da" (UID: "31069c58-7113-4e77-9672-9862fd6576da"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.268833 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.268816    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d93d5ffe-05d3-444c-a816-87c0822c0ce4-kube-api-access-qwjkd" pod "d93d5ffe-05d3-444c-a816-87c0822c0ce4" (UID: "d93d5ffe-05d3-444c-a816-87c0822c0ce4"). InnerVolumeSpecName "kube-api-access-qwjkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:31.271348 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.271292    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-dshm" pod "d93d5ffe-05d3-444c-a816-87c0822c0ce4" (UID: "d93d5ffe-05d3-444c-a816-87c0822c0ce4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:31.271348 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.271292    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tls-certs" pod "d93d5ffe-05d3-444c-a816-87c0822c0ce4" (UID: "d93d5ffe-05d3-444c-a816-87c0822c0ce4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:31.271348 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.271335    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31069c58-7113-4e77-9672-9862fd6576da-tls-certs" pod "31069c58-7113-4e77-9672-9862fd6576da" (UID: "31069c58-7113-4e77-9672-9862fd6576da"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:31.272297 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.272280    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31069c58-7113-4e77-9672-9862fd6576da-kube-api-access-pp94q" pod "31069c58-7113-4e77-9672-9862fd6576da" (UID: "31069c58-7113-4e77-9672-9862fd6576da"). InnerVolumeSpecName "kube-api-access-pp94q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362191    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwjkd\" (UniqueName: \"kubernetes.io/projected/d93d5ffe-05d3-444c-a816-87c0822c0ce4-kube-api-access-qwjkd\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362240    4512 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-model-cache\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362256    4512 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-dshm\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362272    4512 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31069c58-7113-4e77-9672-9862fd6576da-tls-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362284    4512 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-home\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362297    4512 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-model-cache\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362310    4512 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-home\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362323    4512 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tmp-dir\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362336    4512 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d93d5ffe-05d3-444c-a816-87c0822c0ce4-tls-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362349    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp94q\" (UniqueName: \"kubernetes.io/projected/31069c58-7113-4e77-9672-9862fd6576da-kube-api-access-pp94q\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362362    4512 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d93d5ffe-05d3-444c-a816-87c0822c0ce4-dshm\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.363618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.362376    4512 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31069c58-7113-4e77-9672-9862fd6576da-tmp-dir\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.375029 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.374996    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:31.463453 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.463395    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-xks-ca\") pod \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") "
Nov 04 02:22:31.463658 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.463601    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-openshift-ca\") pod \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") "
Nov 04 02:22:31.464069 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.464024    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-openshift-ca" pod "d31b6b0c-f769-4a79-a11f-5689b9deba4d" (UID: "d31b6b0c-f769-4a79-a11f-5689b9deba4d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:31.464181 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.464078    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d31b6b0c-f769-4a79-a11f-5689b9deba4d-kube-api-access-9lgbk\" (UniqueName: \"kubernetes.io/projected/d31b6b0c-f769-4a79-a11f-5689b9deba4d-kube-api-access-9lgbk\") pod \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\" (UID: \"d31b6b0c-f769-4a79-a11f-5689b9deba4d\") "
Nov 04 02:22:31.464564 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.464545    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.466666 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.466323    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-xks-ca" pod "d31b6b0c-f769-4a79-a11f-5689b9deba4d" (UID: "d31b6b0c-f769-4a79-a11f-5689b9deba4d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:31.466923 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.466897    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d31b6b0c-f769-4a79-a11f-5689b9deba4d-kube-api-access-9lgbk" pod "d31b6b0c-f769-4a79-a11f-5689b9deba4d" (UID: "d31b6b0c-f769-4a79-a11f-5689b9deba4d"). InnerVolumeSpecName "kube-api-access-9lgbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:31.565634 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.565599    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lgbk\" (UniqueName: \"kubernetes.io/projected/d31b6b0c-f769-4a79-a11f-5689b9deba4d-kube-api-access-9lgbk\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:31.565852 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:31.565829    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d31b6b0c-f769-4a79-a11f-5689b9deba4d-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:32.106568 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.106466    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"
Nov 04 02:22:32.106568 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.106458    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r" event={"ID":"d93d5ffe-05d3-444c-a816-87c0822c0ce4","Type":"ContainerDied","Data":"4adf35bc391e65487d026b03e28d0e48bdc0f79cfe91e1a05d5babcf55889d83"}
Nov 04 02:22:32.107089 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.106595    4512 scope.go:122] "RemoveContainer" containerID="681cb37b7f865e9f309db1f8127413fd039c4ce36e7663cf1acdb2af832c6bbf"
Nov 04 02:22:32.108167 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.108141    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"
Nov 04 02:22:32.108301 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.108159    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf" event={"ID":"d31b6b0c-f769-4a79-a11f-5689b9deba4d","Type":"ContainerDied","Data":"16da126c002e9c3a00b076069723c6890632b0fa17a5f68c82cc19df43fc2349"}
Nov 04 02:22:32.108439 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.108396    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16da126c002e9c3a00b076069723c6890632b0fa17a5f68c82cc19df43fc2349"
Nov 04 02:22:32.110121 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.110076    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs" event={"ID":"31069c58-7113-4e77-9672-9862fd6576da","Type":"ContainerDied","Data":"a154568ee4234ad0957b8da0ffa8ea316e53e20884903a9bb13e895fc5149c13"}
Nov 04 02:22:32.110121 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.110117    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"
Nov 04 02:22:32.117968 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.117948    4512 scope.go:122] "RemoveContainer" containerID="a48a838e35ec1bb5e612db79ddc8ed7c5545bb7e786b4879c3cc45b84fdd88f7"
Nov 04 02:22:32.273184 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.273150    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"]
Nov 04 02:22:32.322087 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.322056    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-2-w4-196e7e-kserve-669759c6b9-kdgrs"]
Nov 04 02:22:32.388579 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.388496    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"]
Nov 04 02:22:32.391947 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:32.391924    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/simulator-w4-196e7e-kserve-5b9bf6ccdb-nvd4r"]
Nov 04 02:22:33.336796 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:33.336759    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:33.542991 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:33.542957    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:33.908450 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:33.908408    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb" podUID="0aba8252-af93-4aa7-985f-d4164191b6c8" containerName="payload-processing" containerID="cri-o://b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a" gracePeriod=25
Nov 04 02:22:34.019072 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:34.019004    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn" podUID="c7b95100-975a-48f8-b092-678cd9d7168e" containerName="payload-pre-processing" containerID="cri-o://7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8" gracePeriod=25
Nov 04 02:22:34.312961 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:34.312926    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31069c58-7113-4e77-9672-9862fd6576da" path="/var/lib/kubelet/pods/31069c58-7113-4e77-9672-9862fd6576da/volumes"
Nov 04 02:22:34.313363 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:34.313349    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d93d5ffe-05d3-444c-a816-87c0822c0ce4" path="/var/lib/kubelet/pods/d93d5ffe-05d3-444c-a816-87c0822c0ce4/volumes"
Nov 04 02:22:34.502840 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:34.502804    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:38.267266 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:38.267220    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:40.439616 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:40.439571    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:42.359014 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:42.358968    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:43.130967 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:43.130931    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb" podUID="0aba8252-af93-4aa7-985f-d4164191b6c8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:43.131202 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:43.130998    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn" podUID="c7b95100-975a-48f8-b092-678cd9d7168e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:43.184906 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:43.184865    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:44.050319 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:44.050280    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"]
Nov 04 02:22:44.101363 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:44.101324    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-767639c7-zlt8w"]
Nov 04 02:22:44.313746 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:44.313667    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c32c82c1-f62b-4818-862d-91f701414623" path="/var/lib/kubelet/pods/c32c82c1-f62b-4818-862d-91f701414623/volumes"
Nov 04 02:22:48.077998 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:48.077952    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:52.673858 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:52.673821    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:52.932347 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:52.932267    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:53.130536 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:53.130471    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn" podUID="c7b95100-975a-48f8-b092-678cd9d7168e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:53.130536 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:53.130492    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb" podUID="0aba8252-af93-4aa7-985f-d4164191b6c8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:53.141395 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:53.141367    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:53.624295 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:53.624257    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:54.188969 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.188945    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:22:54.225666 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.225632    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0aba8252-af93-4aa7-985f-d4164191b6c8-kube-api-access-p6s9d\" (UniqueName: \"kubernetes.io/projected/0aba8252-af93-4aa7-985f-d4164191b6c8-kube-api-access-p6s9d\") pod \"0aba8252-af93-4aa7-985f-d4164191b6c8\" (UID: \"0aba8252-af93-4aa7-985f-d4164191b6c8\") "
Nov 04 02:22:54.225906 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.225766    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0aba8252-af93-4aa7-985f-d4164191b6c8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aba8252-af93-4aa7-985f-d4164191b6c8-plugins-config-volume\") pod \"0aba8252-af93-4aa7-985f-d4164191b6c8\" (UID: \"0aba8252-af93-4aa7-985f-d4164191b6c8\") "
Nov 04 02:22:54.226220 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.226190    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aba8252-af93-4aa7-985f-d4164191b6c8-plugins-config-volume" pod "0aba8252-af93-4aa7-985f-d4164191b6c8" (UID: "0aba8252-af93-4aa7-985f-d4164191b6c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:54.226359 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.226318    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aba8252-af93-4aa7-985f-d4164191b6c8-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:54.228810 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.228771    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aba8252-af93-4aa7-985f-d4164191b6c8-kube-api-access-p6s9d" pod "0aba8252-af93-4aa7-985f-d4164191b6c8" (UID: "0aba8252-af93-4aa7-985f-d4164191b6c8"). InnerVolumeSpecName "kube-api-access-p6s9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:54.281491 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.281465    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:22:54.293969 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.293945    4512 generic.go:356] "Generic (PLEG): container finished" podID="0aba8252-af93-4aa7-985f-d4164191b6c8" containerID="b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a" exitCode=0
Nov 04 02:22:54.294093 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.293990    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"
Nov 04 02:22:54.294093 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.294033    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb" event={"ID":"0aba8252-af93-4aa7-985f-d4164191b6c8","Type":"ContainerDied","Data":"b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a"}
Nov 04 02:22:54.294209 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.294094    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb" event={"ID":"0aba8252-af93-4aa7-985f-d4164191b6c8","Type":"ContainerDied","Data":"260b9e5e04d5238102aa8ac241579df5b2f0a30d174925caf9871328e609c18d"}
Nov 04 02:22:54.294209 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.294125    4512 scope.go:122] "RemoveContainer" containerID="b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a"
Nov 04 02:22:54.295575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.295554    4512 generic.go:356] "Generic (PLEG): container finished" podID="c7b95100-975a-48f8-b092-678cd9d7168e" containerID="7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8" exitCode=0
Nov 04 02:22:54.295666 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.295600    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"
Nov 04 02:22:54.295666 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.295622    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn" event={"ID":"c7b95100-975a-48f8-b092-678cd9d7168e","Type":"ContainerDied","Data":"7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8"}
Nov 04 02:22:54.295771 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.295682    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn" event={"ID":"c7b95100-975a-48f8-b092-678cd9d7168e","Type":"ContainerDied","Data":"00a0e7bae8ed36b108dc7dfd87d9312c9d449b30643cbf74b1d7a3a726ef21ee"}
Nov 04 02:22:54.308179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.305329    4512 scope.go:122] "RemoveContainer" containerID="b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a"
Nov 04 02:22:54.308179 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:22:54.305644    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a\": container with ID starting with b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a not found: ID does not exist" containerID="b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a"
Nov 04 02:22:54.308179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.305869    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a"} err="failed to get container status \"b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a\": rpc error: code = NotFound desc = could not find container \"b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a\": container with ID starting with b10cbfa766a9daa68b615153dc2d574f15d94b522a768452afb0788cd0ff631a not found: ID does not exist"
Nov 04 02:22:54.308179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.305895    4512 scope.go:122] "RemoveContainer" containerID="7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8"
Nov 04 02:22:54.316868 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.316771    4512 scope.go:122] "RemoveContainer" containerID="7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8"
Nov 04 02:22:54.317114 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:22:54.317086    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8\": container with ID starting with 7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8 not found: ID does not exist" containerID="7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8"
Nov 04 02:22:54.317200 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.317125    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8"} err="failed to get container status \"7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8\": rpc error: code = NotFound desc = could not find container \"7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8\": container with ID starting with 7c448307cc7a5c1eb95ed1a35689bb62caba671e2ca6827e69de8b60f2ccf1f8 not found: ID does not exist"
Nov 04 02:22:54.327365 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.327338    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7b95100-975a-48f8-b092-678cd9d7168e-kube-api-access-xm76z\" (UniqueName: \"kubernetes.io/projected/c7b95100-975a-48f8-b092-678cd9d7168e-kube-api-access-xm76z\") pod \"c7b95100-975a-48f8-b092-678cd9d7168e\" (UID: \"c7b95100-975a-48f8-b092-678cd9d7168e\") "
Nov 04 02:22:54.327471 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.327406    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7b95100-975a-48f8-b092-678cd9d7168e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b95100-975a-48f8-b092-678cd9d7168e-plugins-config-volume\") pod \"c7b95100-975a-48f8-b092-678cd9d7168e\" (UID: \"c7b95100-975a-48f8-b092-678cd9d7168e\") "
Nov 04 02:22:54.327786 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.327763    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7b95100-975a-48f8-b092-678cd9d7168e-plugins-config-volume" pod "c7b95100-975a-48f8-b092-678cd9d7168e" (UID: "c7b95100-975a-48f8-b092-678cd9d7168e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:54.327955 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.327934    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6s9d\" (UniqueName: \"kubernetes.io/projected/0aba8252-af93-4aa7-985f-d4164191b6c8-kube-api-access-p6s9d\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:54.328007 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.327963    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b95100-975a-48f8-b092-678cd9d7168e-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:54.329474 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.329447    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b95100-975a-48f8-b092-678cd9d7168e-kube-api-access-xm76z" pod "c7b95100-975a-48f8-b092-678cd9d7168e" (UID: "c7b95100-975a-48f8-b092-678cd9d7168e"). InnerVolumeSpecName "kube-api-access-xm76z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:54.421490 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.421415    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"]
Nov 04 02:22:54.425135 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.425110    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb"]
Nov 04 02:22:54.428872 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.428848    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm76z\" (UniqueName: \"kubernetes.io/projected/c7b95100-975a-48f8-b092-678cd9d7168e-kube-api-access-xm76z\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:54.716864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.716775    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"]
Nov 04 02:22:54.722841 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:54.722814    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn"]
Nov 04 02:22:56.313065 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:56.313020    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0aba8252-af93-4aa7-985f-d4164191b6c8" path="/var/lib/kubelet/pods/0aba8252-af93-4aa7-985f-d4164191b6c8/volumes"
Nov 04 02:22:56.313642 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:56.313621    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7b95100-975a-48f8-b092-678cd9d7168e" path="/var/lib/kubelet/pods/c7b95100-975a-48f8-b092-678cd9d7168e/volumes"
Nov 04 02:22:56.778997 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:22:56.778957    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:22:57.189990 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:57.189901    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:58.985107 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:58.985072    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:59.382613 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:59.382500    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:59.714851 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:22:59.714756    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:03.883961 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:03.883928    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:05.036131 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:05.036096    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:05.324715 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:05.324621    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:08.048578 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:08.048548    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"]
Nov 04 02:23:08.097476 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:08.097443    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c819d14-g2tf8"]
Nov 04 02:23:08.320732 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:08.320659    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c26fc7d-912d-4d7e-8a21-b7b26942e361" path="/var/lib/kubelet/pods/3c26fc7d-912d-4d7e-8a21-b7b26942e361/volumes"
Nov 04 02:23:08.790924 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:08.790883    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:13.687116 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:13.687082    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:16.669560 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:16.669486    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:17.168430 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:17.168398    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"]
Nov 04 02:23:17.168663 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:17.168638    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy" containerID="cri-o://65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9" gracePeriod=30
Nov 04 02:23:17.227387 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:17.227350    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:18.794761 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:18.794718    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:22.437780 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.437719    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:23:22.504770 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.504740    4512 generic.go:356] "Generic (PLEG): container finished" podID="93902766-a145-4aa9-bd44-e7eb794347b8" containerID="65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9" exitCode=0
Nov 04 02:23:22.504936 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.504786    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"
Nov 04 02:23:22.504936 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.504824    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" event={"ID":"93902766-a145-4aa9-bd44-e7eb794347b8","Type":"ContainerDied","Data":"65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9"}
Nov 04 02:23:22.504936 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.504878    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l" event={"ID":"93902766-a145-4aa9-bd44-e7eb794347b8","Type":"ContainerDied","Data":"6a99eef57e14b8f2bc2ff126247774ed8b4c34bb9d5ac7c9b3b7e29d42e73221"}
Nov 04 02:23:22.504936 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.504905    4512 scope.go:122] "RemoveContainer" containerID="65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9"
Nov 04 02:23:22.514823 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.514804    4512 scope.go:122] "RemoveContainer" containerID="65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9"
Nov 04 02:23:22.515081 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:23:22.515059    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9\": container with ID starting with 65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9 not found: ID does not exist" containerID="65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9"
Nov 04 02:23:22.515150 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.515090    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9"} err="failed to get container status \"65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9\": rpc error: code = NotFound desc = could not find container \"65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9\": container with ID starting with 65c055cc61fce4e93caf10833c95d309060b035e1b67f453b51678737cfffff9 not found: ID does not exist"
Nov 04 02:23:22.551431 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551405    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-socket\") pod \"93902766-a145-4aa9-bd44-e7eb794347b8\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") "
Nov 04 02:23:22.551575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551456    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-envoy\") pod \"93902766-a145-4aa9-bd44-e7eb794347b8\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") "
Nov 04 02:23:22.551575 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551529    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-certs\") pod \"93902766-a145-4aa9-bd44-e7eb794347b8\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") "
Nov 04 02:23:22.551695 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551623    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93902766-a145-4aa9-bd44-e7eb794347b8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93902766-a145-4aa9-bd44-e7eb794347b8-istio-podinfo\") pod \"93902766-a145-4aa9-bd44-e7eb794347b8\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") "
Nov 04 02:23:22.551748 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551690    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-socket" pod "93902766-a145-4aa9-bd44-e7eb794347b8" (UID: "93902766-a145-4aa9-bd44-e7eb794347b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.551748 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551696    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-istio-token\" (UniqueName: \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-istio-token\") pod \"93902766-a145-4aa9-bd44-e7eb794347b8\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") "
Nov 04 02:23:22.551851 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551788    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-certs" pod "93902766-a145-4aa9-bd44-e7eb794347b8" (UID: "93902766-a145-4aa9-bd44-e7eb794347b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.551851 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551804    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-kube-api-access-jzb6s\" (UniqueName: \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-kube-api-access-jzb6s\") pod \"93902766-a145-4aa9-bd44-e7eb794347b8\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") "
Nov 04 02:23:22.551851 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551846    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-data\") pod \"93902766-a145-4aa9-bd44-e7eb794347b8\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") "
Nov 04 02:23:22.552012 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551901    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93902766-a145-4aa9-bd44-e7eb794347b8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93902766-a145-4aa9-bd44-e7eb794347b8-istiod-ca-cert\") pod \"93902766-a145-4aa9-bd44-e7eb794347b8\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") "
Nov 04 02:23:22.552012 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.551945    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-credential-socket\") pod \"93902766-a145-4aa9-bd44-e7eb794347b8\" (UID: \"93902766-a145-4aa9-bd44-e7eb794347b8\") "
Nov 04 02:23:22.552863 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.552304    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-data" pod "93902766-a145-4aa9-bd44-e7eb794347b8" (UID: "93902766-a145-4aa9-bd44-e7eb794347b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.552863 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.552438    4512 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-data\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.552863 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.552459    4512 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.552863 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.552474    4512 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-workload-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.552863 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.552680    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-credential-socket" pod "93902766-a145-4aa9-bd44-e7eb794347b8" (UID: "93902766-a145-4aa9-bd44-e7eb794347b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.552863 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.552827    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93902766-a145-4aa9-bd44-e7eb794347b8-istiod-ca-cert" pod "93902766-a145-4aa9-bd44-e7eb794347b8" (UID: "93902766-a145-4aa9-bd44-e7eb794347b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:23:22.553868 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.553839    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-istio-token" pod "93902766-a145-4aa9-bd44-e7eb794347b8" (UID: "93902766-a145-4aa9-bd44-e7eb794347b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:22.553979 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.553949    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-envoy" pod "93902766-a145-4aa9-bd44-e7eb794347b8" (UID: "93902766-a145-4aa9-bd44-e7eb794347b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:22.554040 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.553991    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-kube-api-access-jzb6s" pod "93902766-a145-4aa9-bd44-e7eb794347b8" (UID: "93902766-a145-4aa9-bd44-e7eb794347b8"). InnerVolumeSpecName "kube-api-access-jzb6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:22.554146 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.554126    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93902766-a145-4aa9-bd44-e7eb794347b8-istio-podinfo" pod "93902766-a145-4aa9-bd44-e7eb794347b8" (UID: "93902766-a145-4aa9-bd44-e7eb794347b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:23:22.653801 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.653770    4512 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93902766-a145-4aa9-bd44-e7eb794347b8-istio-podinfo\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.653801 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.653798    4512 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-istio-token\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.653981 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.653810    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzb6s\" (UniqueName: \"kubernetes.io/projected/93902766-a145-4aa9-bd44-e7eb794347b8-kube-api-access-jzb6s\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.653981 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.653824    4512 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93902766-a145-4aa9-bd44-e7eb794347b8-istiod-ca-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.653981 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.653836    4512 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-credential-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.653981 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.653848    4512 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93902766-a145-4aa9-bd44-e7eb794347b8-istio-envoy\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:22.927135 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.927106    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"]
Nov 04 02:23:22.931454 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:22.931425    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l"]
Nov 04 02:23:24.089392 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:24.089271    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:24.312417 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:24.312385    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" path="/var/lib/kubelet/pods/93902766-a145-4aa9-bd44-e7eb794347b8/volumes"
Nov 04 02:23:26.777980 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:23:26.777942    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:23:28.963627 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:28.963586    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:33.695640 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:33.695600    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:33.901911 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:33.901879    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:37.476642 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.476614    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"]
Nov 04 02:23:37.477171 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.476779    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp" podUID="f1133414-79e3-4a43-b32e-680c0a115b30" containerName="discovery" containerID="cri-o://e62d1549aed963a7b0a4cf8183b25f5c10745c04eaea5d2d1889feb812388c1b" gracePeriod=30
Nov 04 02:23:37.609221 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.609189    4512 generic.go:356] "Generic (PLEG): container finished" podID="f1133414-79e3-4a43-b32e-680c0a115b30" containerID="e62d1549aed963a7b0a4cf8183b25f5c10745c04eaea5d2d1889feb812388c1b" exitCode=0
Nov 04 02:23:37.609338 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.609263    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp" event={"ID":"f1133414-79e3-4a43-b32e-680c0a115b30","Type":"ContainerDied","Data":"e62d1549aed963a7b0a4cf8183b25f5c10745c04eaea5d2d1889feb812388c1b"}
Nov 04 02:23:37.735609 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.735585    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:23:37.785075 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.785048    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-kubeconfig\") pod \"f1133414-79e3-4a43-b32e-680c0a115b30\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") "
Nov 04 02:23:37.785182 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.785126    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-dns-cert\") pod \"f1133414-79e3-4a43-b32e-680c0a115b30\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") "
Nov 04 02:23:37.785182 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.785149    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-kube-api-access-wgvh8\" (UniqueName: \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-kube-api-access-wgvh8\") pod \"f1133414-79e3-4a43-b32e-680c0a115b30\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") "
Nov 04 02:23:37.785182 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.785168    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-ca-configmap\") pod \"f1133414-79e3-4a43-b32e-680c0a115b30\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") "
Nov 04 02:23:37.785301 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.785206    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1133414-79e3-4a43-b32e-680c0a115b30-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1133414-79e3-4a43-b32e-680c0a115b30-local-certs\") pod \"f1133414-79e3-4a43-b32e-680c0a115b30\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") "
Nov 04 02:23:37.785301 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.785267    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-cacerts\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-cacerts\") pod \"f1133414-79e3-4a43-b32e-680c0a115b30\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") "
Nov 04 02:23:37.785400 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.785311    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-istio-token\" (UniqueName: \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-istio-token\") pod \"f1133414-79e3-4a43-b32e-680c0a115b30\" (UID: \"f1133414-79e3-4a43-b32e-680c0a115b30\") "
Nov 04 02:23:37.785705 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.785591    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-ca-configmap" pod "f1133414-79e3-4a43-b32e-680c0a115b30" (UID: "f1133414-79e3-4a43-b32e-680c0a115b30"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:23:37.785797 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.785724    4512 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-ca-configmap\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:37.787387 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.787329    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1133414-79e3-4a43-b32e-680c0a115b30-local-certs" pod "f1133414-79e3-4a43-b32e-680c0a115b30" (UID: "f1133414-79e3-4a43-b32e-680c0a115b30"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:37.787671 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.787633    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-istio-token" pod "f1133414-79e3-4a43-b32e-680c0a115b30" (UID: "f1133414-79e3-4a43-b32e-680c0a115b30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:37.787901 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.787868    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-kubeconfig" pod "f1133414-79e3-4a43-b32e-680c0a115b30" (UID: "f1133414-79e3-4a43-b32e-680c0a115b30"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:23:37.787996 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.787894    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-kube-api-access-wgvh8" pod "f1133414-79e3-4a43-b32e-680c0a115b30" (UID: "f1133414-79e3-4a43-b32e-680c0a115b30"). InnerVolumeSpecName "kube-api-access-wgvh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:37.787996 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.787949    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-cacerts" pod "f1133414-79e3-4a43-b32e-680c0a115b30" (UID: "f1133414-79e3-4a43-b32e-680c0a115b30"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:23:37.788183 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.788164    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-dns-cert" pod "f1133414-79e3-4a43-b32e-680c0a115b30" (UID: "f1133414-79e3-4a43-b32e-680c0a115b30"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:23:37.886772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.886739    4512 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1133414-79e3-4a43-b32e-680c0a115b30-local-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:37.886772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.886768    4512 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-cacerts\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:37.886772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.886778    4512 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-istio-token\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:37.886974 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.886786    4512 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-kubeconfig\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:37.886974 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.886795    4512 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f1133414-79e3-4a43-b32e-680c0a115b30-istio-csr-dns-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:37.886974 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:37.886804    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgvh8\" (UniqueName: \"kubernetes.io/projected/f1133414-79e3-4a43-b32e-680c0a115b30-kube-api-access-wgvh8\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:38.618045 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:38.617973    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"
Nov 04 02:23:38.618045 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:38.617994    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp" event={"ID":"f1133414-79e3-4a43-b32e-680c0a115b30","Type":"ContainerDied","Data":"989f2d05cde92106c25598490a97a2e393eab80cad5f2d8e493b6b30bbb950f6"}
Nov 04 02:23:38.618045 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:38.618045    4512 scope.go:122] "RemoveContainer" containerID="e62d1549aed963a7b0a4cf8183b25f5c10745c04eaea5d2d1889feb812388c1b"
Nov 04 02:23:38.665372 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:38.665333    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:38.833953 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:38.833925    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"]
Nov 04 02:23:38.852620 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:38.852584    4512 scope.go:122] "RemoveContainer" containerID="eb11cbc7feab63b2d991e59f171ea33fa08bcdc2d6c8682c0016b87f36c30647"
Nov 04 02:23:38.862230 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:38.862204    4512 scope.go:122] "RemoveContainer" containerID="3c8c5b1d2fe291ee12bc1b29a90f2033c548e5fa466c3d000bc878efdaa18884"
Nov 04 02:23:38.883779 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:38.883754    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp"]
Nov 04 02:23:40.311973 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:40.311942    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1133414-79e3-4a43-b32e-680c0a115b30" path="/var/lib/kubelet/pods/f1133414-79e3-4a43-b32e-680c0a115b30/volumes"
Nov 04 02:23:43.595843 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:43.595810    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:48.393445 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:48.393414    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:53.375985 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:53.375944    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:56.777772 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:23:56.777730    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:23:58.188255 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:23:58.188134    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:03.072236 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:03.072201    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:04.151714 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:04.151676    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:04.577643 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:04.577607    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:04.776327 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:04.776291    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:07.879447 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:07.879416    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:12.767080 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:12.767032    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:14.049265 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:14.049232    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"]
Nov 04 02:24:14.070022 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:14.069998    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c590b63a-k7qmd"]
Nov 04 02:24:14.312090 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:14.312000    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3" path="/var/lib/kubelet/pods/ddc5f4c8-ab0f-4cc2-970e-4e11c6d785b3/volumes"
Nov 04 02:24:17.565546 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:17.565482    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:17.766647 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:17.766615    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:22.563699 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:22.563662    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:25.051662 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:25.051626    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"]
Nov 04 02:24:25.100708 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:25.100678    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fb9f3d5a-9gmh9"]
Nov 04 02:24:26.311912 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:26.311879    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="885e2d43-195b-4be3-8459-99634df20617" path="/var/lib/kubelet/pods/885e2d43-195b-4be3-8459-99634df20617/volumes"
Nov 04 02:24:26.777248 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:24:26.777208    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:24:27.378033 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:27.378002    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:32.267318 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:32.267274    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:34.042747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:34.042699    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:34.334449 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:34.334361    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:37.088409 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:37.088364    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:38.912745 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:38.912715    4512 scope.go:122] "RemoveContainer" containerID="ed8def9a862cce9bf616ba358d62ea9fcac90319b3d62902a121dc23ebc87528"
Nov 04 02:24:38.922373 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:38.922354    4512 scope.go:122] "RemoveContainer" containerID="34c054b3cde48a6bcda416e5c31f77f1587a551a690213678ed00b2beb05d314"
Nov 04 02:24:42.079450 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:42.079410    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:46.982914 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:46.982873    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:51.878343 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:51.878141    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:53.908412 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:53.908374    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:54.160942 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:54.160838    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:56.782888 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:24:56.782847    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:24:56.783256 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:24:56.782853    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:01.671142 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:01.671106    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:06.497458 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:06.497422    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:11.362738 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:11.362703    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:13.048231 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:13.048200    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"]
Nov 04 02:25:13.097499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:13.097474    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e78b8ff-fvk4h"]
Nov 04 02:25:14.311275 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:14.311239    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ac53138-d0d3-4cf0-ae5a-b0040a825734" path="/var/lib/kubelet/pods/6ac53138-d0d3-4cf0-ae5a-b0040a825734/volumes"
Nov 04 02:25:16.178288 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:16.178240    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:21.069929 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:21.069771    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:22.360041 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:22.360006    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"]
Nov 04 02:25:22.360499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:22.360188    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy" containerID="cri-o://049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3" gracePeriod=30
Nov 04 02:25:25.961255 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:25.961214    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:26.778242 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:25:26.778191    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:25:27.608898 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.608877    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:25:27.706620 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.706552    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-certs\") pod \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") "
Nov 04 02:25:27.706620 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.706592    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-podinfo\") pod \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") "
Nov 04 02:25:27.706767 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.706633    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-token\" (UniqueName: \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-token\") pod \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") "
Nov 04 02:25:27.706767 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.706689    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-envoy\") pod \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") "
Nov 04 02:25:27.706767 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.706736    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-credential-socket\") pod \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") "
Nov 04 02:25:27.706915 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.706781    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istiod-ca-cert\") pod \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") "
Nov 04 02:25:27.706915 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.706841    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-certs" pod "4b19828f-975e-4ed7-8963-29bbb5eb94ca" (UID: "4b19828f-975e-4ed7-8963-29bbb5eb94ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:25:27.706915 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.706860    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-data\") pod \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") "
Nov 04 02:25:27.706915 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.706886    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-kube-api-access-n5p8g\" (UniqueName: \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-kube-api-access-n5p8g\") pod \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") "
Nov 04 02:25:27.707123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.706966    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-socket\") pod \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\" (UID: \"4b19828f-975e-4ed7-8963-29bbb5eb94ca\") "
Nov 04 02:25:27.707123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.707080    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-credential-socket" pod "4b19828f-975e-4ed7-8963-29bbb5eb94ca" (UID: "4b19828f-975e-4ed7-8963-29bbb5eb94ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:25:27.707123 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.707088    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-data" pod "4b19828f-975e-4ed7-8963-29bbb5eb94ca" (UID: "4b19828f-975e-4ed7-8963-29bbb5eb94ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:25:27.707299 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.707274    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-socket" pod "4b19828f-975e-4ed7-8963-29bbb5eb94ca" (UID: "4b19828f-975e-4ed7-8963-29bbb5eb94ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:25:27.707638 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.707451    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istiod-ca-cert" pod "4b19828f-975e-4ed7-8963-29bbb5eb94ca" (UID: "4b19828f-975e-4ed7-8963-29bbb5eb94ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:25:27.707638 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.707457    4512 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-data\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:27.707638 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.707495    4512 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:27.707638 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.707532    4512 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-workload-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:27.707638 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.707545    4512 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-credential-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:27.709058 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.709030    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-envoy" pod "4b19828f-975e-4ed7-8963-29bbb5eb94ca" (UID: "4b19828f-975e-4ed7-8963-29bbb5eb94ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:25:27.709150 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.709131    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-kube-api-access-n5p8g" pod "4b19828f-975e-4ed7-8963-29bbb5eb94ca" (UID: "4b19828f-975e-4ed7-8963-29bbb5eb94ca"). InnerVolumeSpecName "kube-api-access-n5p8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:25:27.709263 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.709241    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-podinfo" pod "4b19828f-975e-4ed7-8963-29bbb5eb94ca" (UID: "4b19828f-975e-4ed7-8963-29bbb5eb94ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:25:27.709356 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.709262    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-token" pod "4b19828f-975e-4ed7-8963-29bbb5eb94ca" (UID: "4b19828f-975e-4ed7-8963-29bbb5eb94ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:25:27.808376 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.808347    4512 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-envoy\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:27.808529 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.808380    4512 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istiod-ca-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:27.808529 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.808398    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5p8g\" (UniqueName: \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-kube-api-access-n5p8g\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:27.808529 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.808413    4512 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-podinfo\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:27.808529 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:27.808427    4512 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b19828f-975e-4ed7-8963-29bbb5eb94ca-istio-token\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:28.355832 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:28.355801    4512 generic.go:356] "Generic (PLEG): container finished" podID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerID="049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3" exitCode=0
Nov 04 02:25:28.355970 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:28.355842    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"
Nov 04 02:25:28.355970 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:28.355874    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" event={"ID":"4b19828f-975e-4ed7-8963-29bbb5eb94ca","Type":"ContainerDied","Data":"049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3"}
Nov 04 02:25:28.355970 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:28.355921    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg" event={"ID":"4b19828f-975e-4ed7-8963-29bbb5eb94ca","Type":"ContainerDied","Data":"79a689160924346dbfdf33566ae903fb670d914d5ce3f4ffb4f74b3fdb7ac057"}
Nov 04 02:25:28.355970 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:28.355942    4512 scope.go:122] "RemoveContainer" containerID="049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3"
Nov 04 02:25:28.365938 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:28.365919    4512 scope.go:122] "RemoveContainer" containerID="049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3"
Nov 04 02:25:28.366226 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:25:28.366208    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3\": container with ID starting with 049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3 not found: ID does not exist" containerID="049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3"
Nov 04 02:25:28.366285 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:28.366233    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3"} err="failed to get container status \"049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3\": rpc error: code = NotFound desc = could not find container \"049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3\": container with ID starting with 049e63f4cd7988098c9199d61257b1c762cbaee70a3bd393dd24cfa39ec587d3 not found: ID does not exist"
Nov 04 02:25:28.572138 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:28.572052    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"]
Nov 04 02:25:28.575297 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:28.575275    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg"]
Nov 04 02:25:30.312592 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:30.312560    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" path="/var/lib/kubelet/pods/4b19828f-975e-4ed7-8963-29bbb5eb94ca/volumes"
Nov 04 02:25:30.797657 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:30.797619    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:33.804069 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:33.804033    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:34.058160 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:34.058019    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:34.398747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:34.398643    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:35.697300 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:35.697253    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:38.748521 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:38.748477    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:25:38.755290 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:38.755252    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:25:38.950915 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:38.950886    4512 scope.go:122] "RemoveContainer" containerID="daa8e0f0e57114a5661e81e66a0afc58495e4a0df1ccaed7b7ddfc4eac567fce"
Nov 04 02:25:38.960333 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:38.960313    4512 scope.go:122] "RemoveContainer" containerID="6809bea27587ab8a9ce1455bfb32e5875bb8caa60e5a85f3af5e8b4d46a56b69"
Nov 04 02:25:40.579150 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:40.579117    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:45.468111 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:45.468076    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:50.295304 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:50.295265    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:55.173336 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:25:55.173295    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:56.777955 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:25:56.777918    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:26:00.077553 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:00.077488    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:04.968366 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:04.968334    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:08.047834 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:08.047799    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"]
Nov 04 02:26:08.097188 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:08.097158    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3dd3c54c-52dsn"]
Nov 04 02:26:08.313672 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:08.313590    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b81dd325-e1d2-4a2f-825d-1228ea1457b8" path="/var/lib/kubelet/pods/b81dd325-e1d2-4a2f-825d-1228ea1457b8/volumes"
Nov 04 02:26:09.966382 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:09.966342    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:14.776855 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:14.776814    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:19.658268 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:19.658174    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:24.486408 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:24.486376    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:26.780980 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:26:26.780940    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:26:29.470240 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:29.470195    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:34.305979 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:34.305940    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:34.767597 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:34.767557    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:38.994521 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:38.994479    4512 scope.go:122] "RemoveContainer" containerID="626aa19f898521702d62e2c12690b75a41c918a73aa2926f5d02fe12afcc0317"
Nov 04 02:26:39.184390 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:39.184287    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:40.050290 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:40.050253    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"]
Nov 04 02:26:40.099513 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:40.099475    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-80f643e6-fp4xt"]
Nov 04 02:26:40.312056 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:40.311988    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ede5ce9b-38db-40d8-bf49-c9b645fbbbfb" path="/var/lib/kubelet/pods/ede5ce9b-38db-40d8-bf49-c9b645fbbbfb/volumes"
Nov 04 02:26:44.071189 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:44.071151    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:49.079039 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:49.079003    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:53.984906 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:53.984868    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:56.777367 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:26:56.777330    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:26:57.048336 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:57.048255    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"]
Nov 04 02:26:57.096960 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:57.096933    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5d4fd4a7-hw8tj"]
Nov 04 02:26:58.049542 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:58.049496    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"]
Nov 04 02:26:58.098774 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:58.098743    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6c0b3916-blgdv"]
Nov 04 02:26:58.311692 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:58.311623    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="689d222a-ebe1-4493-b3ef-08c76a8678b3" path="/var/lib/kubelet/pods/689d222a-ebe1-4493-b3ef-08c76a8678b3/volumes"
Nov 04 02:26:58.312037 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:58.312024    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="872308d5-8d98-49c7-8424-eddc7de82b41" path="/var/lib/kubelet/pods/872308d5-8d98-49c7-8424-eddc7de82b41/volumes"
Nov 04 02:26:58.900698 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:26:58.900660    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:03.768167 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:03.768130    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:08.589876 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:08.589835    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:13.461241 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:13.461144    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:16.048928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:16.048896    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"]
Nov 04 02:27:16.097912 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:16.097882    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a2405010-lpnfs"]
Nov 04 02:27:16.313012 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:16.312921    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a" path="/var/lib/kubelet/pods/ee7edbc9-f82b-404e-9eaf-e5bba4a00f8a/volumes"
Nov 04 02:27:18.372955 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:18.372922    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:22.463702 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.463670    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"]
Nov 04 02:27:22.464224 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464206    4512 cpu_manager.go:414] "containerMap: removing container" podUID="f1133414-79e3-4a43-b32e-680c0a115b30" containerName="discovery"
Nov 04 02:27:22.464265 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464226    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1133414-79e3-4a43-b32e-680c0a115b30" containerName="discovery"
Nov 04 02:27:22.464265 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464235    4512 cpu_manager.go:414] "containerMap: removing container" podUID="d93d5ffe-05d3-444c-a816-87c0822c0ce4" containerName="main"
Nov 04 02:27:22.464265 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464240    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d93d5ffe-05d3-444c-a816-87c0822c0ce4" containerName="main"
Nov 04 02:27:22.464265 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464254    4512 cpu_manager.go:414] "containerMap: removing container" podUID="c7b95100-975a-48f8-b092-678cd9d7168e" containerName="payload-pre-processing"
Nov 04 02:27:22.464265 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464259    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7b95100-975a-48f8-b092-678cd9d7168e" containerName="payload-pre-processing"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464274    4512 cpu_manager.go:414] "containerMap: removing container" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464279    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464302    4512 cpu_manager.go:414] "containerMap: removing container" podUID="31069c58-7113-4e77-9672-9862fd6576da" containerName="main"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464307    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31069c58-7113-4e77-9672-9862fd6576da" containerName="main"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464315    4512 cpu_manager.go:414] "containerMap: removing container" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464320    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464327    4512 cpu_manager.go:414] "containerMap: removing container" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464331    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464337    4512 cpu_manager.go:414] "containerMap: removing container" podUID="0aba8252-af93-4aa7-985f-d4164191b6c8" containerName="payload-processing"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464341    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aba8252-af93-4aa7-985f-d4164191b6c8" containerName="payload-processing"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464348    4512 cpu_manager.go:414] "containerMap: removing container" podUID="d31b6b0c-f769-4a79-a11f-5689b9deba4d" containerName="revoke-keys"
Nov 04 02:27:22.464415 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464352    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d31b6b0c-f769-4a79-a11f-5689b9deba4d" containerName="revoke-keys"
Nov 04 02:27:22.464928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464432    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c7e0fd5-eece-41b5-94e0-a25a09f28332" containerName="istio-proxy"
Nov 04 02:27:22.464928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464445    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="31069c58-7113-4e77-9672-9862fd6576da" containerName="main"
Nov 04 02:27:22.464928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464452    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b19828f-975e-4ed7-8963-29bbb5eb94ca" containerName="istio-proxy"
Nov 04 02:27:22.464928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464458    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="93902766-a145-4aa9-bd44-e7eb794347b8" containerName="istio-proxy"
Nov 04 02:27:22.464928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464464    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aba8252-af93-4aa7-985f-d4164191b6c8" containerName="payload-processing"
Nov 04 02:27:22.464928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464469    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="d31b6b0c-f769-4a79-a11f-5689b9deba4d" containerName="revoke-keys"
Nov 04 02:27:22.464928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464477    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1133414-79e3-4a43-b32e-680c0a115b30" containerName="discovery"
Nov 04 02:27:22.464928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464482    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="d93d5ffe-05d3-444c-a816-87c0822c0ce4" containerName="main"
Nov 04 02:27:22.464928 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.464488    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7b95100-975a-48f8-b092-678cd9d7168e" containerName="payload-pre-processing"
Nov 04 02:27:22.467766 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.467749    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:22.519866 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.519842    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 02:27:22.550729 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.550698    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3f54daad-wx2zd\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:22.550855 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.550762    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3f54daad-wx2zd\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:22.550913 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.550867    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h766s\" (UniqueName: \"kubernetes.io/projected/4756275d-f400-4aa6-ba71-bf6aebd84295-kube-api-access-h766s\") pod \"maas-api-revoke-sub-3f54daad-wx2zd\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:22.569400 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.569379    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\""
Nov 04 02:27:22.572414 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.572395    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"]
Nov 04 02:27:22.651918 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.651891    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3f54daad-wx2zd\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:22.652031 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.651946    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3f54daad-wx2zd\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:22.652031 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.651978    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h766s\" (UniqueName: \"kubernetes.io/projected/4756275d-f400-4aa6-ba71-bf6aebd84295-kube-api-access-h766s\") pod \"maas-api-revoke-sub-3f54daad-wx2zd\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:22.652661 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.652605    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3f54daad-wx2zd\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:22.654332 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.654312    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3f54daad-wx2zd\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:22.711913 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.711885    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h766s\" (UniqueName: \"kubernetes.io/projected/4756275d-f400-4aa6-ba71-bf6aebd84295-kube-api-access-h766s\") pod \"maas-api-revoke-sub-3f54daad-wx2zd\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:22.781235 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:22.781203    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:23.154370 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:23.154340    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"]
Nov 04 02:27:23.155547 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:27:23.155520    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4756275d_f400_4aa6_ba71_bf6aebd84295.slice/crio-cc639243c509eb463370aac890a0bf6fb6c0bf64b410e34e3ff3c0f1ada4eee2 WatchSource:0}: Error finding container cc639243c509eb463370aac890a0bf6fb6c0bf64b410e34e3ff3c0f1ada4eee2: Status 404 returned error can't find the container with id cc639243c509eb463370aac890a0bf6fb6c0bf64b410e34e3ff3c0f1ada4eee2
Nov 04 02:27:23.256277 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:23.256238    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:24.073890 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:24.073859    4512 generic.go:356] "Generic (PLEG): container finished" podID="4756275d-f400-4aa6-ba71-bf6aebd84295" containerID="7c792a75d42ec323af48fa13f5c20d98a7b372d0e836ace3add39fdffe06f170" exitCode=0
Nov 04 02:27:24.074276 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:24.073944    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd" event={"ID":"4756275d-f400-4aa6-ba71-bf6aebd84295","Type":"ContainerDied","Data":"7c792a75d42ec323af48fa13f5c20d98a7b372d0e836ace3add39fdffe06f170"}
Nov 04 02:27:24.074276 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:24.073994    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd" event={"ID":"4756275d-f400-4aa6-ba71-bf6aebd84295","Type":"ContainerStarted","Data":"cc639243c509eb463370aac890a0bf6fb6c0bf64b410e34e3ff3c0f1ada4eee2"}
Nov 04 02:27:25.216314 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:25.216291    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:25.275618 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:25.275573    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-openshift-ca\") pod \"4756275d-f400-4aa6-ba71-bf6aebd84295\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") "
Nov 04 02:27:25.275759 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:25.275656    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-xks-ca\") pod \"4756275d-f400-4aa6-ba71-bf6aebd84295\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") "
Nov 04 02:27:25.275848 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:25.275830    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4756275d-f400-4aa6-ba71-bf6aebd84295-kube-api-access-h766s\" (UniqueName: \"kubernetes.io/projected/4756275d-f400-4aa6-ba71-bf6aebd84295-kube-api-access-h766s\") pod \"4756275d-f400-4aa6-ba71-bf6aebd84295\" (UID: \"4756275d-f400-4aa6-ba71-bf6aebd84295\") "
Nov 04 02:27:25.275925 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:25.275903    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-openshift-ca" pod "4756275d-f400-4aa6-ba71-bf6aebd84295" (UID: "4756275d-f400-4aa6-ba71-bf6aebd84295"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:27:25.276137 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:25.276121    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-openshift-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:25.277892 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:25.277867    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4756275d-f400-4aa6-ba71-bf6aebd84295-kube-api-access-h766s" pod "4756275d-f400-4aa6-ba71-bf6aebd84295" (UID: "4756275d-f400-4aa6-ba71-bf6aebd84295"). InnerVolumeSpecName "kube-api-access-h766s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:27:25.277892 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:25.277878    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-xks-ca" pod "4756275d-f400-4aa6-ba71-bf6aebd84295" (UID: "4756275d-f400-4aa6-ba71-bf6aebd84295"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:27:25.377632 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:25.377563    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h766s\" (UniqueName: \"kubernetes.io/projected/4756275d-f400-4aa6-ba71-bf6aebd84295-kube-api-access-h766s\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:25.377632 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:25.377591    4512 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4756275d-f400-4aa6-ba71-bf6aebd84295-maas-api-xks-ca\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:26.086805 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:26.086775    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd"
Nov 04 02:27:26.086985 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:26.086774    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3f54daad-wx2zd" event={"ID":"4756275d-f400-4aa6-ba71-bf6aebd84295","Type":"ContainerDied","Data":"cc639243c509eb463370aac890a0bf6fb6c0bf64b410e34e3ff3c0f1ada4eee2"}
Nov 04 02:27:26.086985 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:26.086898    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc639243c509eb463370aac890a0bf6fb6c0bf64b410e34e3ff3c0f1ada4eee2"
Nov 04 02:27:26.777124 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:27:26.777092    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:27:28.074922 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:28.074881    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:32.048412 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:32.048378    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"]
Nov 04 02:27:32.097745 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:32.097717    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3c0faf9-9hjxf"]
Nov 04 02:27:32.311470 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:32.311395    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d31b6b0c-f769-4a79-a11f-5689b9deba4d" path="/var/lib/kubelet/pods/d31b6b0c-f769-4a79-a11f-5689b9deba4d/volumes"
Nov 04 02:27:32.956747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:32.956715    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:35.127281 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:35.127234    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:35.375679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:35.375624    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:35.817864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:35.817830    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:36.023674 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:36.023642    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:37.871952 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:37.871920    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:38.114444 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:38.114406    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:38.606840 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:38.606797    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:39.017855 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:39.017822    4512 scope.go:122] "RemoveContainer" containerID="8efe81873214c9d57bebd63a6b68cb4ac707752c903993ef92532d1075c24b96"
Nov 04 02:27:39.029210 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:39.029189    4512 scope.go:122] "RemoveContainer" containerID="a5a99ea287b65a66b22d1853a9bd1330d3a3c622aa8e21a765ea7bea2cb66c76"
Nov 04 02:27:42.838160 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:42.838121    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:43.114330 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:43.114157    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:44.280226 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:44.280182    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:45.025933 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:45.025900    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"]
Nov 04 02:27:45.132684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:45.132643    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"]
Nov 04 02:27:45.446733 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:45.446651    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:45.699974 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:45.699894    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:48.285149 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:48.285099    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:50.026994 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.026826    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh" podUID="a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf" containerName="payload-processing" containerID="cri-o://32f0727ca695d3a8cca867c090bf7e1a2a55112b69dcd28d3c3bc9ca48f2df92" gracePeriod=25
Nov 04 02:27:50.133312 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.133261    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds" podUID="f5084909-e86c-4257-b2b1-7bef33bf0d15" containerName="payload-pre-processing" containerID="cri-o://d22d38d687d6be50f36ae5ab7352c6bc2f936a1111724d4913ea4a3ebbc848e8" gracePeriod=25
Nov 04 02:27:50.300684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.300080    4512 generic.go:356] "Generic (PLEG): container finished" podID="f5084909-e86c-4257-b2b1-7bef33bf0d15" containerID="d22d38d687d6be50f36ae5ab7352c6bc2f936a1111724d4913ea4a3ebbc848e8" exitCode=0
Nov 04 02:27:50.300684 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.300299    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds" event={"ID":"f5084909-e86c-4257-b2b1-7bef33bf0d15","Type":"ContainerDied","Data":"d22d38d687d6be50f36ae5ab7352c6bc2f936a1111724d4913ea4a3ebbc848e8"}
Nov 04 02:27:50.305557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.304155    4512 generic.go:356] "Generic (PLEG): container finished" podID="a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf" containerID="32f0727ca695d3a8cca867c090bf7e1a2a55112b69dcd28d3c3bc9ca48f2df92" exitCode=0
Nov 04 02:27:50.305557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.304260    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh" event={"ID":"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf","Type":"ContainerDied","Data":"32f0727ca695d3a8cca867c090bf7e1a2a55112b69dcd28d3c3bc9ca48f2df92"}
Nov 04 02:27:50.305557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.304301    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh" event={"ID":"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf","Type":"ContainerDied","Data":"3f055e052fde88a98fc947b75bbadef2e67f6ae9f2492ce2cc4992d5b1638b30"}
Nov 04 02:27:50.305557 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.304322    4512 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f055e052fde88a98fc947b75bbadef2e67f6ae9f2492ce2cc4992d5b1638b30"
Nov 04 02:27:50.316806 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.316779    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:27:50.327206 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.327175    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:50.351794 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.350112    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-kube-api-access-q9qbd\" (UniqueName: \"kubernetes.io/projected/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-kube-api-access-q9qbd\") pod \"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf\" (UID: \"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf\") "
Nov 04 02:27:50.351794 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.350163    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-plugins-config-volume\") pod \"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf\" (UID: \"a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf\") "
Nov 04 02:27:50.352056 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.351974    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-plugins-config-volume" pod "a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf" (UID: "a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:27:50.359141 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.359095    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-kube-api-access-q9qbd" pod "a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf" (UID: "a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf"). InnerVolumeSpecName "kube-api-access-q9qbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:27:50.427081 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.427051    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:27:50.450993 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.450948    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5084909-e86c-4257-b2b1-7bef33bf0d15-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5084909-e86c-4257-b2b1-7bef33bf0d15-plugins-config-volume\") pod \"f5084909-e86c-4257-b2b1-7bef33bf0d15\" (UID: \"f5084909-e86c-4257-b2b1-7bef33bf0d15\") "
Nov 04 02:27:50.451187 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.451066    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5084909-e86c-4257-b2b1-7bef33bf0d15-kube-api-access-8m9zc\" (UniqueName: \"kubernetes.io/projected/f5084909-e86c-4257-b2b1-7bef33bf0d15-kube-api-access-8m9zc\") pod \"f5084909-e86c-4257-b2b1-7bef33bf0d15\" (UID: \"f5084909-e86c-4257-b2b1-7bef33bf0d15\") "
Nov 04 02:27:50.451455 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.451436    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9qbd\" (UniqueName: \"kubernetes.io/projected/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-kube-api-access-q9qbd\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:50.451583 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.451445    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5084909-e86c-4257-b2b1-7bef33bf0d15-plugins-config-volume" pod "f5084909-e86c-4257-b2b1-7bef33bf0d15" (UID: "f5084909-e86c-4257-b2b1-7bef33bf0d15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:27:50.451583 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.451461    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:50.453688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.453664    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5084909-e86c-4257-b2b1-7bef33bf0d15-kube-api-access-8m9zc" pod "f5084909-e86c-4257-b2b1-7bef33bf0d15" (UID: "f5084909-e86c-4257-b2b1-7bef33bf0d15"). InnerVolumeSpecName "kube-api-access-8m9zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:27:50.552362 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.552274    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5084909-e86c-4257-b2b1-7bef33bf0d15-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:50.552362 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:50.552314    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m9zc\" (UniqueName: \"kubernetes.io/projected/f5084909-e86c-4257-b2b1-7bef33bf0d15-kube-api-access-8m9zc\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:27:51.312231 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:51.312145    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"
Nov 04 02:27:51.312231 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:51.312192    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds" event={"ID":"f5084909-e86c-4257-b2b1-7bef33bf0d15","Type":"ContainerDied","Data":"9884a1ab9eec9173e7ae834ed9040e63c30f01b58636e873a4fa2ba4d778e5f4"}
Nov 04 02:27:51.312822 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:51.312255    4512 scope.go:122] "RemoveContainer" containerID="d22d38d687d6be50f36ae5ab7352c6bc2f936a1111724d4913ea4a3ebbc848e8"
Nov 04 02:27:51.312822 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:51.312405    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"
Nov 04 02:27:51.518326 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:51.518291    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"]
Nov 04 02:27:51.568016 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:51.567936    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds"]
Nov 04 02:27:51.676812 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:51.676778    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"]
Nov 04 02:27:51.683034 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:51.683003    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh"]
Nov 04 02:27:52.312333 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:52.312300    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf" path="/var/lib/kubelet/pods/a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf/volumes"
Nov 04 02:27:52.312756 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:52.312722    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5084909-e86c-4257-b2b1-7bef33bf0d15" path="/var/lib/kubelet/pods/f5084909-e86c-4257-b2b1-7bef33bf0d15/volumes"
Nov 04 02:27:53.213055 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:53.213001    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:56.778465 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:27:56.778430    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:27:57.476093 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:57.476056    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:59.271820 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:27:59.271786    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:04.204102 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:04.204065    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:04.991250 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:04.991210    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:05.197367 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:05.197324    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:06.608710 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:06.608674    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:07.256464 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:07.256429    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:08.036772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:08.036729    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:09.395098 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:09.395059    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:13.883550 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:13.883481    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:14.222315 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:14.222203    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"]
Nov 04 02:28:14.278758 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:14.278723    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"]
Nov 04 02:28:14.367258 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:14.367220    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:14.563409 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:14.563368    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:15.910973 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:15.910931    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:19.223029 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.222979    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll" podUID="a8fa3273-465a-4741-b8b9-3e98ad56ee46" containerName="payload-processing" containerID="cri-o://801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16" gracePeriod=25
Nov 04 02:28:19.279586 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.279546    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b" podUID="df582a65-4be0-4b85-ae0c-3111b00df6e9" containerName="payload-pre-processing" containerID="cri-o://c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68" gracePeriod=25
Nov 04 02:28:19.500832 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.500800    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:28:19.503591 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.503559    4512 generic.go:356] "Generic (PLEG): container finished" podID="a8fa3273-465a-4741-b8b9-3e98ad56ee46" containerID="801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16" exitCode=0
Nov 04 02:28:19.503753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.503620    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll" event={"ID":"a8fa3273-465a-4741-b8b9-3e98ad56ee46","Type":"ContainerDied","Data":"801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16"}
Nov 04 02:28:19.503753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.503669    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll" event={"ID":"a8fa3273-465a-4741-b8b9-3e98ad56ee46","Type":"ContainerDied","Data":"1267c858021c447211bbbd16780e49cf67cc4cd30f05db7ae41fb1e0f49405e5"}
Nov 04 02:28:19.503753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.503626    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"
Nov 04 02:28:19.503753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.503697    4512 scope.go:122] "RemoveContainer" containerID="801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16"
Nov 04 02:28:19.514156 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.514016    4512 scope.go:122] "RemoveContainer" containerID="801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16"
Nov 04 02:28:19.514437 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:28:19.514409    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16\": container with ID starting with 801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16 not found: ID does not exist" containerID="801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16"
Nov 04 02:28:19.514564 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.514449    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16"} err="failed to get container status \"801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16\": rpc error: code = NotFound desc = could not find container \"801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16\": container with ID starting with 801e42398422c47c603d7742cb6254cf3b851583a6d080d68a4c32db8de0ea16 not found: ID does not exist"
Nov 04 02:28:19.560647 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.560612    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8fa3273-465a-4741-b8b9-3e98ad56ee46-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8fa3273-465a-4741-b8b9-3e98ad56ee46-plugins-config-volume\") pod \"a8fa3273-465a-4741-b8b9-3e98ad56ee46\" (UID: \"a8fa3273-465a-4741-b8b9-3e98ad56ee46\") "
Nov 04 02:28:19.560822 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.560717    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8fa3273-465a-4741-b8b9-3e98ad56ee46-kube-api-access-qh9qp\" (UniqueName: \"kubernetes.io/projected/a8fa3273-465a-4741-b8b9-3e98ad56ee46-kube-api-access-qh9qp\") pod \"a8fa3273-465a-4741-b8b9-3e98ad56ee46\" (UID: \"a8fa3273-465a-4741-b8b9-3e98ad56ee46\") "
Nov 04 02:28:19.560980 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.560958    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8fa3273-465a-4741-b8b9-3e98ad56ee46-plugins-config-volume" pod "a8fa3273-465a-4741-b8b9-3e98ad56ee46" (UID: "a8fa3273-465a-4741-b8b9-3e98ad56ee46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:28:19.562893 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.562867    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8fa3273-465a-4741-b8b9-3e98ad56ee46-kube-api-access-qh9qp" pod "a8fa3273-465a-4741-b8b9-3e98ad56ee46" (UID: "a8fa3273-465a-4741-b8b9-3e98ad56ee46"). InnerVolumeSpecName "kube-api-access-qh9qp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:28:19.661567 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.661523    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8fa3273-465a-4741-b8b9-3e98ad56ee46-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:19.661567 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.661558    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh9qp\" (UniqueName: \"kubernetes.io/projected/a8fa3273-465a-4741-b8b9-3e98ad56ee46-kube-api-access-qh9qp\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:19.724921 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.724821    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b" podUID="df582a65-4be0-4b85-ae0c-3111b00df6e9" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:28:19.925330 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.925279    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"]
Nov 04 02:28:19.930891 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:19.930857    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll"]
Nov 04 02:28:20.286493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:20.286453    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:20.313026 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:20.312990    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8fa3273-465a-4741-b8b9-3e98ad56ee46" path="/var/lib/kubelet/pods/a8fa3273-465a-4741-b8b9-3e98ad56ee46/volumes"
Nov 04 02:28:20.532995 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:20.532945    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:21.155821 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:21.155784    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:21.852140 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:21.852103    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:22.328147 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:22.328096    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:22.785004 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:22.784954    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:26.469220 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:26.469184    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:26.779402 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:28:26.779367    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:28:27.852717 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:27.852679    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:29.724616 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:29.724579    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b" podUID="df582a65-4be0-4b85-ae0c-3111b00df6e9" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:28:29.833444 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:29.833405    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:31.367299 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:31.367258    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:35.044918 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:35.044883    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"]
Nov 04 02:28:35.153749 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:35.153716    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"]
Nov 04 02:28:35.482979 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:35.482945    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:35.768273 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:35.768183    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:36.281861 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:36.281828    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:39.061334 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.061298    4512 scope.go:122] "RemoveContainer" containerID="0a11a6937d9dfaed6c031fb55c4940ebcab592fd561b2fb8650226e83d130502"
Nov 04 02:28:39.072004 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.071979    4512 scope.go:122] "RemoveContainer" containerID="af115a15f78e35a5a2aecc42b291e4d61ae5b9d6a338131767e072f6c3c70720"
Nov 04 02:28:39.080599 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.080574    4512 scope.go:122] "RemoveContainer" containerID="749654ae58f210ddd66a7affba6990bfd14134289913d3ced4cc341be43b90fc"
Nov 04 02:28:39.089898 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.089874    4512 scope.go:122] "RemoveContainer" containerID="ff10794673a17e788e7a32ea9302a28775f5fb9bc9247dfeb6e757f1bfcf5789"
Nov 04 02:28:39.103664 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.101661    4512 scope.go:122] "RemoveContainer" containerID="32f0727ca695d3a8cca867c090bf7e1a2a55112b69dcd28d3c3bc9ca48f2df92"
Nov 04 02:28:39.544411 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.544382    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:28:39.638056 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.638020    4512 generic.go:356] "Generic (PLEG): container finished" podID="df582a65-4be0-4b85-ae0c-3111b00df6e9" containerID="c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68" exitCode=0
Nov 04 02:28:39.638267 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.638067    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"
Nov 04 02:28:39.638267 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.638086    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b" event={"ID":"df582a65-4be0-4b85-ae0c-3111b00df6e9","Type":"ContainerDied","Data":"c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68"}
Nov 04 02:28:39.638267 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.638169    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b" event={"ID":"df582a65-4be0-4b85-ae0c-3111b00df6e9","Type":"ContainerDied","Data":"ec988994ac65791a92ace6255db303b1b057ec442460fc8ef7127f6ff9ecd3d2"}
Nov 04 02:28:39.638267 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.638225    4512 scope.go:122] "RemoveContainer" containerID="c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68"
Nov 04 02:28:39.648084 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.648066    4512 scope.go:122] "RemoveContainer" containerID="c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68"
Nov 04 02:28:39.648339 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:28:39.648321    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68\": container with ID starting with c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68 not found: ID does not exist" containerID="c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68"
Nov 04 02:28:39.648411 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.648352    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68"} err="failed to get container status \"c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68\": rpc error: code = NotFound desc = could not find container \"c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68\": container with ID starting with c122c6a900dcf321d212c0931e0382473fc321589885ab1ceb5a1ebfe592bb68 not found: ID does not exist"
Nov 04 02:28:39.668042 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.667974    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df582a65-4be0-4b85-ae0c-3111b00df6e9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df582a65-4be0-4b85-ae0c-3111b00df6e9-plugins-config-volume\") pod \"df582a65-4be0-4b85-ae0c-3111b00df6e9\" (UID: \"df582a65-4be0-4b85-ae0c-3111b00df6e9\") "
Nov 04 02:28:39.668183 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.668157    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df582a65-4be0-4b85-ae0c-3111b00df6e9-kube-api-access-ldmrq\" (UniqueName: \"kubernetes.io/projected/df582a65-4be0-4b85-ae0c-3111b00df6e9-kube-api-access-ldmrq\") pod \"df582a65-4be0-4b85-ae0c-3111b00df6e9\" (UID: \"df582a65-4be0-4b85-ae0c-3111b00df6e9\") "
Nov 04 02:28:39.668433 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.668408    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df582a65-4be0-4b85-ae0c-3111b00df6e9-plugins-config-volume" pod "df582a65-4be0-4b85-ae0c-3111b00df6e9" (UID: "df582a65-4be0-4b85-ae0c-3111b00df6e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:28:39.670421 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.670390    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df582a65-4be0-4b85-ae0c-3111b00df6e9-kube-api-access-ldmrq" pod "df582a65-4be0-4b85-ae0c-3111b00df6e9" (UID: "df582a65-4be0-4b85-ae0c-3111b00df6e9"). InnerVolumeSpecName "kube-api-access-ldmrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:28:39.769679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.769637    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df582a65-4be0-4b85-ae0c-3111b00df6e9-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:39.769679 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:39.769674    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldmrq\" (UniqueName: \"kubernetes.io/projected/df582a65-4be0-4b85-ae0c-3111b00df6e9-kube-api-access-ldmrq\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:28:40.045355 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:40.045316    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn" podUID="7c2a1484-282c-49d8-b4d6-43b41d23f18a" containerName="payload-processing" containerID="cri-o://cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658" gracePeriod=25
Nov 04 02:28:40.064206 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:40.064172    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"]
Nov 04 02:28:40.113619 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:40.113582    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b"]
Nov 04 02:28:40.158530 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:40.157577    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x" podUID="a5c9cdc8-5445-40db-8f1e-ae6b61861eca" containerName="payload-pre-processing" containerID="cri-o://60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2" gracePeriod=25
Nov 04 02:28:40.315018 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:40.314979    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df582a65-4be0-4b85-ae0c-3111b00df6e9" path="/var/lib/kubelet/pods/df582a65-4be0-4b85-ae0c-3111b00df6e9/volumes"
Nov 04 02:28:41.979160 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:41.979124    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:47.892752 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:47.892710    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:49.097872 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:49.097830    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn" podUID="7c2a1484-282c-49d8-b4d6-43b41d23f18a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:28:49.100363 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:49.097898    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x" podUID="a5c9cdc8-5445-40db-8f1e-ae6b61861eca" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:28:51.279156 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:51.279111    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:53.522885 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:53.522848    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:53.767809 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:53.767771    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:28:56.778906 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:28:56.778870    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:28:59.097683 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:59.097642    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x" podUID="a5c9cdc8-5445-40db-8f1e-ae6b61861eca" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:28:59.098196 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:28:59.098156    4512 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn" podUID="7c2a1484-282c-49d8-b4d6-43b41d23f18a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:29:00.307475 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.307415    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:29:00.408264 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.408239    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c2a1484-282c-49d8-b4d6-43b41d23f18a-kube-api-access-5kvfr\" (UniqueName: \"kubernetes.io/projected/7c2a1484-282c-49d8-b4d6-43b41d23f18a-kube-api-access-5kvfr\") pod \"7c2a1484-282c-49d8-b4d6-43b41d23f18a\" (UID: \"7c2a1484-282c-49d8-b4d6-43b41d23f18a\") "
Nov 04 02:29:00.408387 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.408276    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c2a1484-282c-49d8-b4d6-43b41d23f18a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a1484-282c-49d8-b4d6-43b41d23f18a-plugins-config-volume\") pod \"7c2a1484-282c-49d8-b4d6-43b41d23f18a\" (UID: \"7c2a1484-282c-49d8-b4d6-43b41d23f18a\") "
Nov 04 02:29:00.408675 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.408651    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c2a1484-282c-49d8-b4d6-43b41d23f18a-plugins-config-volume" pod "7c2a1484-282c-49d8-b4d6-43b41d23f18a" (UID: "7c2a1484-282c-49d8-b4d6-43b41d23f18a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:29:00.410337 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.410312    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2a1484-282c-49d8-b4d6-43b41d23f18a-kube-api-access-5kvfr" pod "7c2a1484-282c-49d8-b4d6-43b41d23f18a" (UID: "7c2a1484-282c-49d8-b4d6-43b41d23f18a"). InnerVolumeSpecName "kube-api-access-5kvfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:29:00.417493 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.417473    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:29:00.509322 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.509299    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-plugins-config-volume\") pod \"a5c9cdc8-5445-40db-8f1e-ae6b61861eca\" (UID: \"a5c9cdc8-5445-40db-8f1e-ae6b61861eca\") "
Nov 04 02:29:00.509457 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.509383    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-kube-api-access-5plqz\" (UniqueName: \"kubernetes.io/projected/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-kube-api-access-5plqz\") pod \"a5c9cdc8-5445-40db-8f1e-ae6b61861eca\" (UID: \"a5c9cdc8-5445-40db-8f1e-ae6b61861eca\") "
Nov 04 02:29:00.509688 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.509667    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-plugins-config-volume" pod "a5c9cdc8-5445-40db-8f1e-ae6b61861eca" (UID: "a5c9cdc8-5445-40db-8f1e-ae6b61861eca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:29:00.509742 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.509729    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kvfr\" (UniqueName: \"kubernetes.io/projected/7c2a1484-282c-49d8-b4d6-43b41d23f18a-kube-api-access-5kvfr\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:00.509790 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.509750    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a1484-282c-49d8-b4d6-43b41d23f18a-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:00.511616 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.511596    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-kube-api-access-5plqz" pod "a5c9cdc8-5445-40db-8f1e-ae6b61861eca" (UID: "a5c9cdc8-5445-40db-8f1e-ae6b61861eca"). InnerVolumeSpecName "kube-api-access-5plqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:29:00.610568 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.610480    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5plqz\" (UniqueName: \"kubernetes.io/projected/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-kube-api-access-5plqz\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:00.610568 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.610531    4512 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5c9cdc8-5445-40db-8f1e-ae6b61861eca-plugins-config-volume\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:00.782558 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.782493    4512 generic.go:356] "Generic (PLEG): container finished" podID="a5c9cdc8-5445-40db-8f1e-ae6b61861eca" containerID="60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2" exitCode=0
Nov 04 02:29:00.782737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.782567    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"
Nov 04 02:29:00.782737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.782581    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x" event={"ID":"a5c9cdc8-5445-40db-8f1e-ae6b61861eca","Type":"ContainerDied","Data":"60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2"}
Nov 04 02:29:00.782737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.782642    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x" event={"ID":"a5c9cdc8-5445-40db-8f1e-ae6b61861eca","Type":"ContainerDied","Data":"b65fac4f126f3d566d672c148530b3191e9630a6899013f7cbae1077b2bbc9b4"}
Nov 04 02:29:00.782737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.782670    4512 scope.go:122] "RemoveContainer" containerID="60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2"
Nov 04 02:29:00.784087 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.784065    4512 generic.go:356] "Generic (PLEG): container finished" podID="7c2a1484-282c-49d8-b4d6-43b41d23f18a" containerID="cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658" exitCode=0
Nov 04 02:29:00.784195 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.784114    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"
Nov 04 02:29:00.784195 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.784138    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn" event={"ID":"7c2a1484-282c-49d8-b4d6-43b41d23f18a","Type":"ContainerDied","Data":"cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658"}
Nov 04 02:29:00.784195 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.784171    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn" event={"ID":"7c2a1484-282c-49d8-b4d6-43b41d23f18a","Type":"ContainerDied","Data":"65895400b02671e3e63809c875f80fa12e8e06b234b0cfcccd46b855dd5d34b7"}
Nov 04 02:29:00.791968 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.791954    4512 scope.go:122] "RemoveContainer" containerID="60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2"
Nov 04 02:29:00.792234 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:29:00.792214    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2\": container with ID starting with 60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2 not found: ID does not exist" containerID="60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2"
Nov 04 02:29:00.792304 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.792243    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2"} err="failed to get container status \"60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2\": rpc error: code = NotFound desc = could not find container \"60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2\": container with ID starting with 60cd16431fd110a566d1e494e85a6d7cece5b0e635835eb16618d748d65243e2 not found: ID does not exist"
Nov 04 02:29:00.792304 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.792258    4512 scope.go:122] "RemoveContainer" containerID="cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658"
Nov 04 02:29:00.804278 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.804263    4512 scope.go:122] "RemoveContainer" containerID="cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658"
Nov 04 02:29:00.804544 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:29:00.804522    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658\": container with ID starting with cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658 not found: ID does not exist" containerID="cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658"
Nov 04 02:29:00.804612 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.804552    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658"} err="failed to get container status \"cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658\": rpc error: code = NotFound desc = could not find container \"cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658\": container with ID starting with cdab2ade5515abd38144393811d908d8f71727fa102cae2ccc75e1381e573658 not found: ID does not exist"
Nov 04 02:29:00.950866 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.950775    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"]
Nov 04 02:29:00.953797 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:00.953778    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn"]
Nov 04 02:29:01.064732 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:01.064712    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"]
Nov 04 02:29:01.067347 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:01.067315    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x"]
Nov 04 02:29:02.312330 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:02.312286    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c2a1484-282c-49d8-b4d6-43b41d23f18a" path="/var/lib/kubelet/pods/7c2a1484-282c-49d8-b4d6-43b41d23f18a/volumes"
Nov 04 02:29:02.312853 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:02.312831    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5c9cdc8-5445-40db-8f1e-ae6b61861eca" path="/var/lib/kubelet/pods/a5c9cdc8-5445-40db-8f1e-ae6b61861eca/volumes"
Nov 04 02:29:04.227879 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:04.227843    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:29:04.442829 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:04.442793    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:29:05.276327 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:05.276281    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:29:05.481179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:05.481146    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:29:07.388256 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388226    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"]
Nov 04 02:29:07.388729 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388715    4512 cpu_manager.go:414] "containerMap: removing container" podUID="a8fa3273-465a-4741-b8b9-3e98ad56ee46" containerName="payload-processing"
Nov 04 02:29:07.388772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388731    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8fa3273-465a-4741-b8b9-3e98ad56ee46" containerName="payload-processing"
Nov 04 02:29:07.388772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388755    4512 cpu_manager.go:414] "containerMap: removing container" podUID="a5c9cdc8-5445-40db-8f1e-ae6b61861eca" containerName="payload-pre-processing"
Nov 04 02:29:07.388772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388764    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5c9cdc8-5445-40db-8f1e-ae6b61861eca" containerName="payload-pre-processing"
Nov 04 02:29:07.388864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388780    4512 cpu_manager.go:414] "containerMap: removing container" podUID="df582a65-4be0-4b85-ae0c-3111b00df6e9" containerName="payload-pre-processing"
Nov 04 02:29:07.388864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388785    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df582a65-4be0-4b85-ae0c-3111b00df6e9" containerName="payload-pre-processing"
Nov 04 02:29:07.388864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388793    4512 cpu_manager.go:414] "containerMap: removing container" podUID="f5084909-e86c-4257-b2b1-7bef33bf0d15" containerName="payload-pre-processing"
Nov 04 02:29:07.388864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388798    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5084909-e86c-4257-b2b1-7bef33bf0d15" containerName="payload-pre-processing"
Nov 04 02:29:07.388864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388804    4512 cpu_manager.go:414] "containerMap: removing container" podUID="4756275d-f400-4aa6-ba71-bf6aebd84295" containerName="revoke-keys"
Nov 04 02:29:07.388864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388808    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4756275d-f400-4aa6-ba71-bf6aebd84295" containerName="revoke-keys"
Nov 04 02:29:07.388864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388818    4512 cpu_manager.go:414] "containerMap: removing container" podUID="7c2a1484-282c-49d8-b4d6-43b41d23f18a" containerName="payload-processing"
Nov 04 02:29:07.388864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388822    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c2a1484-282c-49d8-b4d6-43b41d23f18a" containerName="payload-processing"
Nov 04 02:29:07.388864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388848    4512 cpu_manager.go:414] "containerMap: removing container" podUID="a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf" containerName="payload-processing"
Nov 04 02:29:07.388864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388853    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf" containerName="payload-processing"
Nov 04 02:29:07.389148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388928    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="a25e6a97-5cac-4bbd-88d0-d1c9f7aad4cf" containerName="payload-processing"
Nov 04 02:29:07.389148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388937    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8fa3273-465a-4741-b8b9-3e98ad56ee46" containerName="payload-processing"
Nov 04 02:29:07.389148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388944    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5c9cdc8-5445-40db-8f1e-ae6b61861eca" containerName="payload-pre-processing"
Nov 04 02:29:07.389148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388952    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="df582a65-4be0-4b85-ae0c-3111b00df6e9" containerName="payload-pre-processing"
Nov 04 02:29:07.389148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388958    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5084909-e86c-4257-b2b1-7bef33bf0d15" containerName="payload-pre-processing"
Nov 04 02:29:07.389148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388963    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="4756275d-f400-4aa6-ba71-bf6aebd84295" containerName="revoke-keys"
Nov 04 02:29:07.389148 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.388969    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c2a1484-282c-49d8-b4d6-43b41d23f18a" containerName="payload-processing"
Nov 04 02:29:07.392474 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.392459    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.444572 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.444548    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-29e7e8-openshift-default-dockercfg-7ntvz\""
Nov 04 02:29:07.478867 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.478834    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.479041 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.478882    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.479041 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.478967    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.479041 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.479016    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.479226 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.479046    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.479226 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.479109    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.479226 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.479142    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.479407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.479327    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.479475 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.479407    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmhgq\" (UniqueName: \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-kube-api-access-zmhgq\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.497145 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.497114    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"]
Nov 04 02:29:07.580152 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580122    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580304 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580156    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580304 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580215    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580304 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580252    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580490 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580383    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580490 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580467    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580637 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580500    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580637 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580596    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580632    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580648    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmhgq\" (UniqueName: \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-kube-api-access-zmhgq\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580747 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580651    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580889 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580804    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.580946 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.580916    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.581098 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.581073    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.582895 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.582873    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.582895 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.582892    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.643086 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.643024    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.643722 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.643703    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmhgq\" (UniqueName: \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-kube-api-access-zmhgq\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:07.706036 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:07.706013    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:08.032855 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:08.032823    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:29:08.133489 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:08.133463    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"]
Nov 04 02:29:08.135647 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:29:08.135620    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e834d90_1a7c_456f_8fcc_74e2d4f1b9b6.slice/crio-8a09f028274b3bc8b1b0c5dd6f2f955962da6f5b50b275400551a908c5986a93 WatchSource:0}: Error finding container 8a09f028274b3bc8b1b0c5dd6f2f955962da6f5b50b275400551a908c5986a93: Status 404 returned error can't find the container with id 8a09f028274b3bc8b1b0c5dd6f2f955962da6f5b50b275400551a908c5986a93
Nov 04 02:29:08.137749 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:08.137717    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:29:08.137794 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:08.137782    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:29:08.137842 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:08.137814    4512 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 04 02:29:08.844166 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:08.844130    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" event={"ID":"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6","Type":"ContainerStarted","Data":"bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10"}
Nov 04 02:29:08.844598 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:08.844195    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" event={"ID":"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6","Type":"ContainerStarted","Data":"8a09f028274b3bc8b1b0c5dd6f2f955962da6f5b50b275400551a908c5986a93"}
Nov 04 02:29:09.019040 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:09.018907    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" podStartSLOduration=2.018888414 podStartE2EDuration="2.018888414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:29:09.01834525 +0000 UTC m=+2010.823801883" watchObservedRunningTime="2026-11-04 02:29:09.018888414 +0000 UTC m=+2010.824345048"
Nov 04 02:29:09.710607 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:09.710568    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:09.711636 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:09.711609    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Nov 04 02:29:09.711846 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:09.711820    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" podUID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Nov 04 02:29:10.706912 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:10.706872    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Nov 04 02:29:10.707931 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:10.707901    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" podUID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Nov 04 02:29:11.362441 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:11.362403    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:29:11.613984 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:11.613883    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:29:11.708886 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:11.708850    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Nov 04 02:29:11.709529 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:11.709470    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" podUID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Nov 04 02:29:12.707531 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:12.706852    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Nov 04 02:29:12.707531 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:12.706930    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" podUID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Nov 04 02:29:13.709531 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:13.708916    4512 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Nov 04 02:29:13.709531 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:13.708995    4512 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" podUID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Nov 04 02:29:14.710213 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:14.710175    4512 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:14.710640 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:14.710566    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:14.711480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:14.711462    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:18.535301 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:18.535271    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"]
Nov 04 02:29:18.535768 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:18.535461    4512 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" podUID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" containerName="istio-proxy" containerID="cri-o://bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10" gracePeriod=30
Nov 04 02:29:18.588772 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:18.588744    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:29:19.475720 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:19.475685    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86c994f77b-qwzgv_0c66053c-2382-4412-93ca-f67f1274fd08/authorino/0.log"
Nov 04 02:29:19.889952 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:19.889921    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-tx25t_848ca595-eb11-49f4-bf0d-77d303fb0432/manager/0.log"
Nov 04 02:29:21.403219 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:21.403185    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-g65d7_c557873e-e31e-4447-8ba1-e8a84c3cc5f6/manager/0.log"
Nov 04 02:29:22.084579 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:22.084548    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-htz9j_3802223a-2462-41a3-98ef-884d9dcc36f2/manager/0.log"
Nov 04 02:29:23.389222 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.389196    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-67t5h_a712b8d8-6477-411b-ab42-a925a4a2e397/manager/0.log"
Nov 04 02:29:23.626499 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.626473    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jtkx7_35c6a7e4-1ffb-45d1-86ed-9707bf91b699/manager/0.log"
Nov 04 02:29:23.790083 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.790057    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:23.860324 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860301    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-envoy\") pod \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") "
Nov 04 02:29:23.860453 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860336    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-socket\") pod \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") "
Nov 04 02:29:23.860453 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860405    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istiod-ca-cert\") pod \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") "
Nov 04 02:29:23.860453 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860427    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-credential-socket\") pod \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") "
Nov 04 02:29:23.860659 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860482    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-kube-api-access-zmhgq\" (UniqueName: \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-kube-api-access-zmhgq\") pod \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") "
Nov 04 02:29:23.860659 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860541    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-podinfo\") pod \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") "
Nov 04 02:29:23.860659 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860599    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-certs\") pod \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") "
Nov 04 02:29:23.860788 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860659    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-token\" (UniqueName: \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-token\") pod \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") "
Nov 04 02:29:23.860788 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860681    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-socket" pod "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" (UID: "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:29:23.860788 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860710    4512 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-data\") pod \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\" (UID: \"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6\") "
Nov 04 02:29:23.860969 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.860954    4512 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.862424 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.862403    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-envoy" pod "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" (UID: "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:29:23.863849 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.863803    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-credential-socket" pod "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" (UID: "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:29:23.863968 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.863820    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istiod-ca-cert" pod "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" (UID: "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:29:23.863968 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.863846    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-certs" pod "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" (UID: "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:29:23.863968 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.863859    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-data" pod "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" (UID: "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:29:23.865632 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.865600    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-kube-api-access-zmhgq" pod "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" (UID: "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6"). InnerVolumeSpecName "kube-api-access-zmhgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:29:23.865632 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.865604    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-podinfo" pod "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" (UID: "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:29:23.865766 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.865723    4512 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-token" pod "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" (UID: "2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:29:23.942941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.942871    4512 generic.go:356] "Generic (PLEG): container finished" podID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" containerID="bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10" exitCode=0
Nov 04 02:29:23.943059 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.942939    4512 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"
Nov 04 02:29:23.943059 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.942970    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" event={"ID":"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6","Type":"ContainerDied","Data":"bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10"}
Nov 04 02:29:23.943154 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.943058    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8" event={"ID":"2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6","Type":"ContainerDied","Data":"8a09f028274b3bc8b1b0c5dd6f2f955962da6f5b50b275400551a908c5986a93"}
Nov 04 02:29:23.943154 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.943089    4512 scope.go:122] "RemoveContainer" containerID="bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10"
Nov 04 02:29:23.953094 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.953062    4512 scope.go:122] "RemoveContainer" containerID="bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10"
Nov 04 02:29:23.953459 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:29:23.953432    4512 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10\": container with ID starting with bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10 not found: ID does not exist" containerID="bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10"
Nov 04 02:29:23.953581 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.953466    4512 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10"} err="failed to get container status \"bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10\": rpc error: code = NotFound desc = could not find container \"bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10\": container with ID starting with bb02d64328ada65c2ce1899fc5b150b163b545935d8f95d095cceec1e2f02e10 not found: ID does not exist"
Nov 04 02:29:23.962108 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.962084    4512 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-workload-certs\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.962186 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.962113    4512 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-token\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.962186 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.962126    4512 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-data\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.962186 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.962139    4512 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-envoy\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.962186 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.962152    4512 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istiod-ca-cert\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.962186 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.962166    4512 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-credential-socket\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.962186 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.962179    4512 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmhgq\" (UniqueName: \"kubernetes.io/projected/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-kube-api-access-zmhgq\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:23.962404 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:23.962201    4512 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6-istio-podinfo\") on node \"ip-10-0-142-191.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:29:24.108115 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:24.108083    4512 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"]
Nov 04 02:29:24.111972 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:24.111951    4512 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8"]
Nov 04 02:29:24.241063 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:24.241040    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-575cb5b89b-xd5wg_aaa16c33-a697-4a91-b30b-68fb63c959c5/manager/1.log"
Nov 04 02:29:24.311744 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:24.311717    4512 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" path="/var/lib/kubelet/pods/2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6/volumes"
Nov 04 02:29:24.489494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:24.489471    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jvj68_9cb40c10-cfc5-4d9e-954f-462e1ba04257/server/0.log"
Nov 04 02:29:24.963995 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:24.963953    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-xb2fb_97a87324-2f8e-42fb-99cf-13583bb87171/proxy/0.log"
Nov 04 02:29:25.615904 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:25.615869    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-575cb5b89b-xd5wg_aaa16c33-a697-4a91-b30b-68fb63c959c5/manager/0.log"
Nov 04 02:29:26.777833 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:29:26.777799    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:29:29.591667 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:29.591641    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86c994f77b-qwzgv_0c66053c-2382-4412-93ca-f67f1274fd08/authorino/0.log"
Nov 04 02:29:35.836191 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:29:35.836143    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:29:56.777822 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:29:56.777785    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:30:02.129738 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:02.129705    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-67t5h_a712b8d8-6477-411b-ab42-a925a4a2e397/manager/0.log"
Nov 04 02:30:02.370501 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:02.370469    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jtkx7_35c6a7e4-1ffb-45d1-86ed-9707bf91b699/manager/0.log"
Nov 04 02:30:03.031753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:03.031723    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-575cb5b89b-xd5wg_aaa16c33-a697-4a91-b30b-68fb63c959c5/manager/1.log"
Nov 04 02:30:03.296909 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:03.296819    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jvj68_9cb40c10-cfc5-4d9e-954f-462e1ba04257/server/0.log"
Nov 04 02:30:03.836322 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:03.836296    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-xb2fb_97a87324-2f8e-42fb-99cf-13583bb87171/proxy/0.log"
Nov 04 02:30:04.490417 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:04.490388    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-575cb5b89b-xd5wg_aaa16c33-a697-4a91-b30b-68fb63c959c5/manager/0.log"
Nov 04 02:30:15.980799 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:15.980769    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3f54daad-wx2zd_4756275d-f400-4aa6-ba71-bf6aebd84295/revoke-keys/0.log"
Nov 04 02:30:16.463864 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:16.463779    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-s2h6f_4805db53-c126-426f-b0e1-9e051e33c509/postgres/0.log"
Nov 04 02:30:18.747204 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:18.747175    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86c994f77b-qwzgv_0c66053c-2382-4412-93ca-f67f1274fd08/authorino/0.log"
Nov 04 02:30:19.167354 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:19.167231    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-tx25t_848ca595-eb11-49f4-bf0d-77d303fb0432/manager/0.log"
Nov 04 02:30:20.373836 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:20.373804    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-g65d7_c557873e-e31e-4447-8ba1-e8a84c3cc5f6/manager/0.log"
Nov 04 02:30:20.854355 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:20.854314    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-htz9j_3802223a-2462-41a3-98ef-884d9dcc36f2/manager/0.log"
Nov 04 02:30:21.893650 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:21.893619    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk_9bde987c-e9ca-48e4-b60a-e6b394e3ea6e/istio-proxy/0.log"
Nov 04 02:30:26.778403 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:30:26.778370    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:30:34.256036 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:34.255998    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5_907abcc8-9f0d-4809-b0d4-56326e5fe3d8/istio-proxy/0.log"
Nov 04 02:30:34.481850 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:34.481821    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-r74jj_8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2/istio-proxy/1.log"
Nov 04 02:30:34.990930 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:34.990901    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-tnrml_5c820e07-bdfa-43ee-bce8-4d38241f414a/istio-proxy/0.log"
Nov 04 02:30:35.211001 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:35.210973    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-c554b664c-9n829_e2bbd606-cb7b-49c8-883d-aa3ae68d5d17/istio-proxy/0.log"
Nov 04 02:30:36.201723 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:36.201683    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:30:36.419112 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:36.419075    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:30:36.658962 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:36.658931    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-r74jj_8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2/istio-proxy/0.log"
Nov 04 02:30:37.246675 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:37.246650    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl_6fe316ae-8d04-40f0-a6ca-7c4e71893b86/main/0.log"
Nov 04 02:30:37.750026 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:37.749998    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-b77xj_e18368a3-6ffc-481d-a7fe-011065a9a24a/main/0.log"
Nov 04 02:30:38.459984 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:38.459947    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-k7wrt_b11d4503-32db-4aaf-a651-78fc2b44c413/main/0.log"
Nov 04 02:30:38.695869 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:38.695833    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5_c17897b5-2085-4013-aeb3-2b55b6503572/main/0.log"
Nov 04 02:30:38.791073 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:38.791047    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:30:38.800297 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:38.800272    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aklprj_d5de9d8b-4ca3-488a-a727-54f0204173d2/extract/0.log"
Nov 04 02:30:49.529852 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:49.529765    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:30:52.538472 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:52.538435    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:30:56.776911 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:30:56.776874    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:30:58.482865 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:30:58.482833    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dlxgx_51998d39-62ca-432f-bac3-3b8142d05e0e/global-pull-secret-syncer/0.log"
Nov 04 02:31:01.655939 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:01.655911    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log"
Nov 04 02:31:03.646787 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:03.646751    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:31:03.761060 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:03.761026    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-191.us-west-1.compute.internal_9f88ddff8fbb738ea2fba74409a8de90/haproxy/0.log"
Nov 04 02:31:03.844099 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:03.844063    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:31:08.572759 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.572722    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86c994f77b-qwzgv_0c66053c-2382-4412-93ca-f67f1274fd08/authorino/0.log"
Nov 04 02:31:08.768144 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.768105    4512 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"]
Nov 04 02:31:08.768686 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.768670    4512 cpu_manager.go:414] "containerMap: removing container" podUID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" containerName="istio-proxy"
Nov 04 02:31:08.768737 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.768688    4512 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" containerName="istio-proxy"
Nov 04 02:31:08.768798 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.768788    4512 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e834d90-1a7c-456f-8fcc-74e2d4f1b9b6" containerName="istio-proxy"
Nov 04 02:31:08.772051 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.772036    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.824616 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.824533    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w2sfr\"/\"openshift-service-ca.crt\""
Nov 04 02:31:08.874368 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.874319    4512 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w2sfr\"/\"kube-root-ca.crt\""
Nov 04 02:31:08.874368 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.874328    4512 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w2sfr\"/\"default-dockercfg-lvbsf\""
Nov 04 02:31:08.879140 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.879113    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-sys\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.879239 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.879168    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"]
Nov 04 02:31:08.879280 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.879169    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-proc\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.879314 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.879280    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-lib-modules\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.879423 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.879403    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-podres\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.879500 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.879485    4512 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4zhb\" (UniqueName: \"kubernetes.io/projected/8cda8689-9d2d-4b05-92e5-f3580cbc5545-kube-api-access-b4zhb\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.895383 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.895345    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-tx25t_848ca595-eb11-49f4-bf0d-77d303fb0432/manager/0.log"
Nov 04 02:31:08.980299 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.980257    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-podres\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.980494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.980318    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4zhb\" (UniqueName: \"kubernetes.io/projected/8cda8689-9d2d-4b05-92e5-f3580cbc5545-kube-api-access-b4zhb\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.980494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.980350    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-sys\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.980494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.980391    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-proc\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.980494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.980408    4512 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-lib-modules\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.980494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.980405    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-podres\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.980494 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.980476    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-sys\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.980749 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.980493    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-proc\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:08.980749 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:08.980497    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cda8689-9d2d-4b05-92e5-f3580cbc5545-lib-modules\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:09.041077 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:09.041036    4512 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4zhb\" (UniqueName: \"kubernetes.io/projected/8cda8689-9d2d-4b05-92e5-f3580cbc5545-kube-api-access-b4zhb\") pod \"perf-node-gather-daemonset-dc7s8\" (UID: \"8cda8689-9d2d-4b05-92e5-f3580cbc5545\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:09.085964 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:09.085871    4512 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:09.467547 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:09.467311    4512 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"]
Nov 04 02:31:09.470645 ip-10-0-142-191 kubenswrapper[4512]: W1104 02:31:09.470603    4512 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8cda8689_9d2d_4b05_92e5_f3580cbc5545.slice/crio-8009b81de1a8dced81aa4da459cda3f4a720e71cc5eeae56dec0129f9e5b2cc4 WatchSource:0}: Error finding container 8009b81de1a8dced81aa4da459cda3f4a720e71cc5eeae56dec0129f9e5b2cc4: Status 404 returned error can't find the container with id 8009b81de1a8dced81aa4da459cda3f4a720e71cc5eeae56dec0129f9e5b2cc4
Nov 04 02:31:09.557569 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:09.557534    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8" event={"ID":"8cda8689-9d2d-4b05-92e5-f3580cbc5545","Type":"ContainerStarted","Data":"8009b81de1a8dced81aa4da459cda3f4a720e71cc5eeae56dec0129f9e5b2cc4"}
Nov 04 02:31:10.568010 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:10.567962    4512 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8" event={"ID":"8cda8689-9d2d-4b05-92e5-f3580cbc5545","Type":"ContainerStarted","Data":"2fa50199f4c77bd3d2b5fc9af21f4363637811e445043a8620348f6b7972c96a"}
Nov 04 02:31:10.568567 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:10.568091    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:10.648298 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:10.648264    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-g65d7_c557873e-e31e-4447-8ba1-e8a84c3cc5f6/manager/0.log"
Nov 04 02:31:10.685830 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:10.685779    4512 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8" podStartSLOduration=2.685762672 podStartE2EDuration="2.685762672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:31:10.684665432 +0000 UTC m=+2132.490122101" watchObservedRunningTime="2026-11-04 02:31:10.685762672 +0000 UTC m=+2132.491219302"
Nov 04 02:31:11.441480 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:11.441444    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-htz9j_3802223a-2462-41a3-98ef-884d9dcc36f2/manager/0.log"
Nov 04 02:31:12.086693 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:12.086658    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-67t5h_a712b8d8-6477-411b-ab42-a925a4a2e397/manager/0.log"
Nov 04 02:31:12.317693 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:12.317649    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jtkx7_35c6a7e4-1ffb-45d1-86ed-9707bf91b699/manager/0.log"
Nov 04 02:31:12.772429 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:12.772396    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-575cb5b89b-xd5wg_aaa16c33-a697-4a91-b30b-68fb63c959c5/manager/0.log"
Nov 04 02:31:12.875179 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:12.875137    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-575cb5b89b-xd5wg_aaa16c33-a697-4a91-b30b-68fb63c959c5/manager/1.log"
Nov 04 02:31:12.983383 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:12.983339    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jvj68_9cb40c10-cfc5-4d9e-954f-462e1ba04257/server/0.log"
Nov 04 02:31:13.740713 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:13.740679    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-xb2fb_97a87324-2f8e-42fb-99cf-13583bb87171/proxy/0.log"
Nov 04 02:31:14.059725 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:14.059636    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d12e508-96a3-4551-84e5-da9736a50368/alertmanager/0.log"
Nov 04 02:31:14.479799 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:14.479708    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d12e508-96a3-4551-84e5-da9736a50368/config-reloader/0.log"
Nov 04 02:31:14.898753 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:14.898723    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d12e508-96a3-4551-84e5-da9736a50368/kube-rbac-proxy-web/0.log"
Nov 04 02:31:15.317873 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:15.317837    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d12e508-96a3-4551-84e5-da9736a50368/kube-rbac-proxy/0.log"
Nov 04 02:31:15.738063 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:15.738032    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d12e508-96a3-4551-84e5-da9736a50368/kube-rbac-proxy-metric/0.log"
Nov 04 02:31:16.158680 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:16.158596    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d12e508-96a3-4551-84e5-da9736a50368/prom-label-proxy/0.log"
Nov 04 02:31:16.578407 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:16.578374    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d12e508-96a3-4551-84e5-da9736a50368/init-config-reloader/0.log"
Nov 04 02:31:16.585038 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:16.585017    4512 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-dc7s8"
Nov 04 02:31:16.703473 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:16.703435    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-67t5h_a712b8d8-6477-411b-ab42-a925a4a2e397/manager/0.log"
Nov 04 02:31:16.936841 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:16.936764    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jtkx7_35c6a7e4-1ffb-45d1-86ed-9707bf91b699/manager/0.log"
Nov 04 02:31:17.392898 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:17.392857    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-575cb5b89b-xd5wg_aaa16c33-a697-4a91-b30b-68fb63c959c5/manager/0.log"
Nov 04 02:31:17.405719 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:17.405689    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:31:17.438738 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:17.438714    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-7zdv8_c6137457-318e-414f-9351-7ccf6ed64333/kube-state-metrics/0.log"
Nov 04 02:31:17.498960 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:17.498931    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-575cb5b89b-xd5wg_aaa16c33-a697-4a91-b30b-68fb63c959c5/manager/1.log"
Nov 04 02:31:17.611563 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:17.611500    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jvj68_9cb40c10-cfc5-4d9e-954f-462e1ba04257/server/0.log"
Nov 04 02:31:17.615389 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:17.615365    4512 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5v5x7_dfca312e-b151-4c3a-99bd-dd820925264b/konnectivity-agent/0.log" limit=1024
Nov 04 02:31:17.845578 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:17.845545    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-7zdv8_c6137457-318e-414f-9351-7ccf6ed64333/kube-rbac-proxy-main/0.log"
Nov 04 02:31:18.266361 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:18.266336    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-7zdv8_c6137457-318e-414f-9351-7ccf6ed64333/kube-rbac-proxy-self/0.log"
Nov 04 02:31:18.405241 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:18.405203    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-xb2fb_97a87324-2f8e-42fb-99cf-13583bb87171/proxy/0.log"
Nov 04 02:31:18.690628 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:18.690541    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5584fc7458-wcht4_a71b41b2-6ae5-47cb-b2e6-6df4a70f2919/metrics-server/0.log"
Nov 04 02:31:19.115293 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:19.115264    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-zp926_db8099f3-2446-4394-907b-5c30917a084f/monitoring-plugin/0.log"
Nov 04 02:31:20.808262 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:20.808231    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9dlvz_0a557337-43bd-42f2-809c-1ded60fbff48/node-exporter/0.log"
Nov 04 02:31:21.226185 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:21.226098    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9dlvz_0a557337-43bd-42f2-809c-1ded60fbff48/kube-rbac-proxy/0.log"
Nov 04 02:31:21.368658 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:21.368626    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-67t5h_a712b8d8-6477-411b-ab42-a925a4a2e397/manager/0.log"
Nov 04 02:31:21.606098 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:21.606069    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jtkx7_35c6a7e4-1ffb-45d1-86ed-9707bf91b699/manager/0.log"
Nov 04 02:31:21.646919 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:21.646893    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9dlvz_0a557337-43bd-42f2-809c-1ded60fbff48/init-textfile/0.log"
Nov 04 02:31:22.060983 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:22.060948    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-575cb5b89b-xd5wg_aaa16c33-a697-4a91-b30b-68fb63c959c5/manager/0.log"
Nov 04 02:31:22.159933 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:22.159901    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-575cb5b89b-xd5wg_aaa16c33-a697-4a91-b30b-68fb63c959c5/manager/1.log"
Nov 04 02:31:22.270267 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:22.270231    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-jvj68_9cb40c10-cfc5-4d9e-954f-462e1ba04257/server/0.log"
Nov 04 02:31:23.088958 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:23.088921    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-xb2fb_97a87324-2f8e-42fb-99cf-13583bb87171/proxy/0.log"
Nov 04 02:31:23.346438 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:23.346345    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-zbq9m_3d10c3f0-fede-4fec-bdbc-bfdfdf4788df/kube-rbac-proxy-main/0.log"
Nov 04 02:31:23.769880 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:23.769849    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-zbq9m_3d10c3f0-fede-4fec-bdbc-bfdfdf4788df/kube-rbac-proxy-self/0.log"
Nov 04 02:31:24.189059 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:24.188971    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-zbq9m_3d10c3f0-fede-4fec-bdbc-bfdfdf4788df/openshift-state-metrics/0.log"
Nov 04 02:31:26.331249 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:26.331212    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk_9bde987c-e9ca-48e4-b60a-e6b394e3ea6e/istio-proxy/0.log"
Nov 04 02:31:26.784441 ip-10-0-142-191 kubenswrapper[4512]: E1104 02:31:26.784393    4512 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-s2nfk" containerName="extractor"
Nov 04 02:31:28.451062 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:28.451034    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-2526s_0f437497-7f45-4df4-a309-7bdd9bb697c4/prometheus-operator-admission-webhook/0.log"
Nov 04 02:31:29.243124 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:29.243092    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5_907abcc8-9f0d-4809-b0d4-56326e5fe3d8/istio-proxy/0.log"
Nov 04 02:31:29.457369 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:29.457335    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-r74jj_8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2/istio-proxy/0.log"
Nov 04 02:31:29.459966 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:29.459946    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-r74jj_8ec5e90c-2c2e-40ae-98eb-2ff62195a4a2/istio-proxy/1.log"
Nov 04 02:31:29.783385 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:29.783346    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-tnrml_5c820e07-bdfa-43ee-bce8-4d38241f414a/istio-proxy/0.log"
Nov 04 02:31:29.995867 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:29.995831    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-c554b664c-9n829_e2bbd606-cb7b-49c8-883d-aa3ae68d5d17/istio-proxy/0.log"
Nov 04 02:31:30.146573 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:30.146471    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fbfb964d-m4zw5_afeff4a9-d88a-4cd6-9127-7531da99f4af/thanos-query/0.log"
Nov 04 02:31:30.566359 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:30.566328    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fbfb964d-m4zw5_afeff4a9-d88a-4cd6-9127-7531da99f4af/kube-rbac-proxy-web/0.log"
Nov 04 02:31:30.992648 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:30.992623    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fbfb964d-m4zw5_afeff4a9-d88a-4cd6-9127-7531da99f4af/kube-rbac-proxy/0.log"
Nov 04 02:31:31.411941 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:31.411862    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fbfb964d-m4zw5_afeff4a9-d88a-4cd6-9127-7531da99f4af/prom-label-proxy/0.log"
Nov 04 02:31:31.832185 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:31.832147    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fbfb964d-m4zw5_afeff4a9-d88a-4cd6-9127-7531da99f4af/kube-rbac-proxy-rules/0.log"
Nov 04 02:31:32.256442 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:32.256402    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fbfb964d-m4zw5_afeff4a9-d88a-4cd6-9127-7531da99f4af/kube-rbac-proxy-metrics/0.log"
Nov 04 02:31:32.609189 ip-10-0-142-191 kubenswrapper[4512]: I1104 02:31:32.609158    4512 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk_9bde987c-e9ca-48e4-b60a-e6b394e3ea6e/istio-proxy/0.log"